A method and system for drone-assisted vehicle-to-everything (V2X) data transmission

Abstract

This invention relates to the field of intelligent systems technology for drone scheduling, trajectory optimization, and vehicle-to-everything (V2X) communication, specifically to a method and system for drone-assisted V2X data transmission. The method includes: randomly deploying multiple drones at 5G base stations within a task area; a server receiving data requests from vehicle cluster heads within the task area, making scheduling decisions using a cloud-based drone scheduling model deployed on the server, and sending the target vehicle cluster head ID corresponding to the selected drone to that drone; each drone, upon receiving its target vehicle cluster head ID, planning its flight path using a flight path planning model deployed on its own drone; and upon reaching the target vehicle cluster head, the drone establishing a wireless transmission network between the vehicle and the macro base station using its onboard 5G micro base station and CPE equipment, and transmitting data. Through this method, this invention forms a V2X data transmission system that enables multi-drone, multi-base station collaboration.

Description

Technical Field

[0001] This invention relates to the field of intelligent system technology for drone scheduling, trajectory optimization, and vehicle networking, and specifically to a method and system for drone-assisted vehicle networking data transmission. Background Technology

[0002] In recent years, the increasing maturity of intelligent transportation and internet technologies has spawned many emerging vehicle applications, such as autonomous driving, in-vehicle entertainment, and data sharing. These applications share a common characteristic: they generally have very high requirements for transmission latency, transmission quality, and transmission speed. While the rapidly developing 5G communication technology can meet the communication needs of intelligent transportation, the communication range of 5G base stations is often only one or two hundred meters, and the construction and maintenance costs are high. These factors prevent the deployment and coverage of 5G base stations from being as widespread as cellular networks. Using 5G base stations to cover 5G networks in areas where vehicles only occasionally pass by would result in a serious waste of resources. Furthermore, in some remote areas, stable 4G network transmission still faces problems due to obstruction by buildings and vegetation.

[0003] The maturity of drone technology has led to a new paradigm for drone-assisted vehicle-to-everything (V2X) communication. Drones are less expensive to deploy than 5G base stations, and due to their higher altitudes, they can simultaneously establish line-of-sight (LoS) connections with both vehicles and base stations. Utilizing drones for communication can significantly improve wireless network transmission efficiency. At the 72nd 3GPPTSG-RAN plenary meeting, eight networking methods for communication were proposed. Among them, the non-standalone (NSA) networking method using a 5G core network-enhanced 4G base station-5G base station can improve transmission speed and also increase the transmission range for drone-assisted communication. On the other hand, significant challenges remain regarding drone scheduling, trajectory optimization, coordination between 5G base stations and drones, as well as the cloud, and the complete automation of drone-assisted V2X communication.

[0004] The existing problems with drone-assisted vehicle-to-everything (V2X) communication are as follows: (1) A fully automated drone-assisted V2X data transmission system with end-edge-cloud collaboration has not been formed. (2) The drone's flight maneuvers cannot be dynamically adjusted according to the vehicle's location to meet the data transmission needs of different traffic conditions. (3) A V2X data transmission system with multiple drones and multiple base stations has not been formed. Summary of the Invention

[0005] To address the above problems, this invention proposes a method for unmanned aerial vehicle (UAV)-assisted vehicle-to-everything (V2X) data transmission.

[0006] The method includes:

[0007] Step 1: Randomly deploy multiple drones at 5G base stations within the mission area. Each drone is equipped with a 5G micro base station and a CPE device.

[0008] Step 2: The server receives data requests from vehicle cluster heads within the task area.

[0009] If the vehicle cluster head is connected to a 5G base station, it will request data through the 5G base station.

[0010] If the vehicle fails to connect to the 5G base station, it requests data from the server via the cellular network;

[0011] Step 3: The server integrates data request information from all 5G base stations and vehicles, uses the cloud-based drone scheduling model deployed on the server to make scheduling decisions based on the status information of base stations, vehicles, and drones, and sends the target vehicle cluster head ID corresponding to the selected drone to the drone.

[0012] The cloud-based drone scheduling model is a pre-trained decision model aimed at maximizing system data throughput, which includes the data throughput of 5G base stations and the data throughput of drones.

[0013] Step 4: After each UAV receives its target vehicle cluster head ID, it accesses the real-time location of the target vehicle cluster head from the server based on the ID, and uses the flight path planning model deployed on the UAV itself to plan the UAV's flight path based on its own status information and the real-time location of the target vehicle cluster head.

[0014] The unmanned flight route planning model is a pre-trained decision model aimed at maximizing the data throughput of the unmanned aerial vehicle.

[0015] Step 5: After the drone arrives at the head of the target vehicle cluster, it uses its onboard 5G micro base station and CPE equipment to establish a wireless transmission network between the vehicle and the macro base station and transmit data.

[0016] Furthermore, it also includes step 6: After the drone-assisted transmission mission is completed, it flies back to any 5G base station in the mission area to park and charge.

[0017] The unmanned flight route planning model is also used to plan the route for the drone to fly back to a certain 5G base station based on its own status information and the location of the 5G base station after the drone has completed the task of assisting in transmission.

[0018] Furthermore, the status information of the base station and the vehicle includes the location of the base station and the location of the vehicle cluster head that accesses the 5G base station;

[0019] Furthermore, the drone status information includes the drone's location, battery level, and whether the drone is currently performing a mission.

[0020] Furthermore, the cloud-based drone scheduling model adopts a DQN model;

[0021] Furthermore, the task area is gridded into multiple map grids, each of which is uniquely numbered.

[0022] The state space of the DQN model used for cloud-based drone scheduling is:

[0023] State = {B Ip D Ip E j ,S j V Ip},

[0024] Among them, B Ip Let K be the locations of 5G base stations in a gridded map, with a dimension of 1×K;

[0025] D Ip Let J be the positions of J drones in a gridded map, with dimension 1×J;

[0026] V Ip The location of at most I vehicle cluster heads that are not connected to a 5G base station in the gridded map, with a dimension of 1×1;

[0027] The B Ip D Ip V Ip The location in the text is the grid number where the 5G base station, drone, or vehicle cluster head is located;

[0028] Vehicle clusters that fail to access a 5G base station within a decision-making cycle are numbered by the server as 1, 2...I, corresponding to vector V. Ip The first to the ith element, each value being the map grid number where the corresponding vehicle cluster head was located when it issued the request. If the number of vehicle cluster heads that have failed to connect to the 5G base station is less than ith, then -1 is used to fill the gap to ith. Ip Items in;

[0029] E j The value represents the power of the drone, with a dimension of 1×J, where the element value ranges from 0 to 1, and the closer to 1, the more energy is available;

[0030] UAV current mission status S j The dimension is 1×J, where the element value ranges from 0 to 1, where 0 means not executing a task and 1 means executing a task.

[0031] The DQN model used for cloud-based drone scheduling has a network action space dimension of J. The output value of each dimension is mapped to an integer between 0 and 1, representing the target vehicle cluster head number of the drone, so as to obtain the position of the vehicle cluster head at any time. A value of 0 indicates that the drone has not been assigned any vehicle cluster head. The action space can be represented as follows:

[0032] Action = {a0, a1, a2, ..., a...} J}, a j ∈[0, 1, 2, ..., I].

[0033] Furthermore, the reward function used to train the DQN model for cloud-based drone scheduling is:

[0034] Reward = k r D total -r p ,

[0035] Where k r =0.01 is a constant coefficient, and the penalty term r p When the output action space contains a0, a1, a2, ..., a J After mapping to integers 0 to 1, if there are duplicate numbers other than 0, then r... p =1, otherwise r p =0.

[0036] Furthermore, the decision output of the cloud-based drone scheduling model is the drone's action in the next moment, including moving forward, backward, upward, downward, left, right, or hovering for a period of time;

[0037] Drones form an optimized trajectory by performing a series of actions in succession.

[0038] Furthermore, the drone is equipped with a lidar for sensing obstacle information, and the drone scheduling model also plans the drone's flight route based on the obstacle information.

[0039] The model is trained under the constraint of not colliding with obstacles.

[0040] Furthermore, the unmanned flight route planning model adopts a DQN model;

[0041] Furthermore, the state space of the DQN model used for unmanned flight path planning is as follows:

[0042] S={V p E(n), D p O s B Ip},

[0043] Among them, V p The target vehicle cluster head's three-dimensional position dimension is 3;

[0044] E(n) represents the drone's energy at the current moment, with dimension 1, n = 0, 1, 2...N, indicating that the energy is expressed in terms of time t. N The discrete time is the time slot length;

[0045] D p This indicates the three-dimensional position of the drone in front, with a dimension of 3;

[0046] O s O represents the obstacle information currently sensed by the lidar. s A fixed 20-dimensional list is used to represent the distribution of obstacles around the drone, and its configuration strategy is as follows:

[0047] Within a certain detection distance centered on the drone, ten directional angle intervals evenly distributed on the horizontal plane and ten directional angle intervals evenly distributed on the vertical plane correspond to O respectively. s The 20 dimensions, each directional angle interval has a width of 360° / 10 = 36°. A value of 1 indicates that there is an obstacle in that interval, and a value of 0 indicates that there is no obstacle.

[0048] The action space of the model's output is:

[0049] A={left, right, forward, backward, up, down, hover},

[0050] When left is 1, it means to move left; when right is 1, it means to move right; when forward is 1, it means to move forward; when backward is 1, it means to move backward; when up is 1, it means to rise; when down is 1, it means to fall; when hover is 1, it means to hover. Only one variable will change to 1 at any given time.

[0051] Furthermore, the reward function for training the DQN model used for unmanned flight path planning is:

[0052]

[0053] Where, k th R total (n) represents the reward for the data throughput of the drone in time slot n, k th =0.00001 is a constant weighting coefficient. To shape the rewards for the drone's interaction with the target point and energy, and to prevent reward sparsity during training, krd = 0.5 is a constant weighting coefficient. Used to prevent the denominator from being zero. d represents the energy consumed within a time slot. ais r represents the reduction in distance between the UAV and the destination within the current time slot. c The reward for a drone landing at a base station, r ad This represents the penalty for a drone collision;

[0054] The total data throughput of the UAV within time slot n is:

[0055] R total (n)=R i (nt N +t0)t N ;

[0056] Where t0 is the mission initiation time, and the transmission rate between the UAV and vehicle cluster head i at time t is expressed as:

[0057]

[0058] Among them, P c The communication power d for the UAV to provide communication services to a single vehicle cluster head i i Let n be the distance between vehicle cluster head i and UAV, n0 be the noise power spectral density, and α be the minimum communication radius of UAV.

[0059] Where L i (t) represents the path loss between vehicle cluster head i and the UAV, and we have:

[0060]

[0061] Where spl represents the spatial path loss. The probability of connecting at the Loss of Sight (LOS). The probability of NLoS (non-line-of-sight) connection, and η LoS With η NLoS The transitional path loss between Los and NLoS connections;

[0062] Based on the drone's altitude And drones can calculate the probability of line-of-sight connectivity:

[0063]

[0064] Where η a η b To transmit environmental parameters, Where f c For the carrier frequency, v c The speed of light;

[0065] The model should also meet the following constraints during training:

[0066] C1: E T >0

[0067] C2:

[0068] C3: CL = 0

[0069] C4:x min <x<x max

[0070] C5:y min <y<y max

[0071] C6:z min <z<z max ;

[0072] E in C1 T This indicates the drone's energy level at the end of the mission;

[0073] D in C2 pT B represents the three-dimensional position of the UAV at the end of the mission. P A set representing the three-dimensional locations of base stations;

[0074] In C3, CL represents the number of collisions;

[0075] In C4-C6, x, y, and z represent the three-dimensional coordinates of the UAV;

[0076] During training and decision-making, the input data for this model is pre-set according to the following strategy:

[0077] When the drone is performing a mission, force setting B Ip All values ​​in the table are -1;

[0078] When the drone is en route back to the base station, the input V is forcibly set. p The value is (0, -1, 0).

[0079] Another objective of this invention is to provide a drone-assisted vehicle-to-everything (V2X) data transmission system, comprising a 5G base station, a cloud server, and a drone;

[0080] The drone is equipped with a positioning module and a wireless transmission module;

[0081] The 5G base station, cloud server, and drone are interconnected.

[0082] The system assists in the data transmission of the vehicle network using the aforementioned method.

[0083] The method in this invention comprises two layers: the first layer is a cloud-based decision-making layer, which includes a cloud server for scheduling multiple drones and assigning corresponding tasks to each drone; the second layer is a drone execution layer, where drones execute tasks from the cloud and perform auxiliary data transmission. The optimization objective of the first cloud-based scheduling layer is to maximize the overall system's data throughput, while the optimization objective of the second drone behavior decision-making layer is to maximize the drone's data throughput, forming a fully automated, edge-cloud collaborative drone-assisted vehicle-to-everything (V2X) data transmission system. It can dynamically adjust the drone's flight maneuvers based on the position of the vehicle cluster head to meet the data transmission needs of different traffic conditions, thus forming a multi-drone, multi-base station collaborative V2X data transmission system. Attached Figure Description

[0084] Figure 1 This is an architecture diagram of the UAV-assisted vehicle-to-everything (V2X) data transmission system in an embodiment of the present invention;

[0085] Figure 2 This is a flowchart of the UAV-assisted vehicle-to-everything (V2X) data transmission method in an embodiment of the present invention;

[0086] Figure 3 This is an example of generating obstacle information O in this invention. s A schematic diagram of spatial division. Detailed Implementation

[0087] The embodiments of the present invention are described in detail below according to the system architecture diagram and system flowchart. The system architecture diagram is only used as an example, and it can be extended based on this architecture in practical applications.

[0088] like Figure 1As shown, this embodiment addresses a scenario with multiple 5G base stations but incomplete coverage, specifically a road segment with few 5G base stations. The scenario includes a few 5G base stations covering areas with high traffic volume. Multiple drones are deployed in some of these base stations. When a vehicle travels in an area not yet covered by the 5G network and requires significant high-speed data transmission, the vehicle requests a drone from the server to assist with data transmission. The server responds to the vehicle's request and decides to dispatch a drone to assist the vehicle in data transmission. All base stations include both basic communication modules and drone charging modules. In this application scenario, the vehicle is a smart transportation vehicle equipped with various sensors, including GPS locators and communication modules, which can transmit vehicle location information and data request information to the cloud server and drones. The drones are equipped with 5G micro base stations and CPE (Customer Premise Equipment) devices to establish a wireless transmission network between the vehicle and the macro base station and transmit data. Additionally, each drone is equipped with a LiDAR, which periodically scans the surrounding environment, including the location and direction vectors of obstacles.

[0089] The drone is also equipped with a wireless transmission module, a computing unit, a storage unit, and a positioning module; the wireless transmission module is used for communication between the drone and vehicles and 5G base stations; the computing unit is used for online decision-making calculations by the drone; the storage unit is used to store data; and the positioning module is used to provide real-time feedback on the drone's specific location.

[0090] In this embodiment, the task area is first gridded. The length and width of each grid can be adjusted according to the performance of the UAV and the total area of ​​the scheduling area. In this embodiment, the scheduling area is a square area of ​​10000m×10000m. The task area is divided into grids with a length and width of 100m and all grids are numbered with integers from 0 to 9999. The positions of the vehicle and the 5G base station are mapped to the corresponding grid numbers. The scheduling layer uses an integer to represent the location access of the vehicle, UAV and base station.

[0091] In this embodiment, a two-layer decision model is constructed: a cloud-based drone scheduling layer for drone scheduling and allocation decisions; and a drone trajectory optimization layer for drone flight action decisions.

[0092] The cloud-based drone scheduling model is deployed on a server. This model makes decisions based on the location of the vehicle cluster head, the drone's location, the drone's energy, the base station location, and the drone's status, and schedules drones to go to the mission area to assist vehicles in transmitting data.

[0093] The cloud-based drone scheduling model in this embodiment is a DQN decision model based on deep reinforcement learning. In this decision model, K 5G base stations, J drones, and the maximum number of vehicle cluster heads accessed are defined as I according to the actual situation of the task area.

[0094] This decision model integrates request data from vehicles and 5G base stations with the optimization objective of maximizing the data throughput of the entire system.

[0095] The system throughput consists of two parts. One part is the amount of data directly forwarded to the vehicle by the 5G base station. The data throughput of all 5G base stations during the entire operation is defined as follows:

[0096]

[0097] Among them, Rb total (t) represents the transmission rate of the base station.

[0098] The other part is the data throughput achieved through data transmission assisted by drones, expressed as:

[0099]

[0100] Where R j，total This represents the total data transmission volume of drone j;

[0101] Therefore, the optimization objective of the decision model is to maximize the overall system throughput.

[0102] Pi: Max(B total +D total ).

[0103] The reinforcement learning-based DQN model proposed in this embodiment is used to solve the above problem.

[0104] Therefore, the state space of the DQN model includes:

[0105] Location B of K 5G base stations on the gridded map Ip The dimension is 1×K;

[0106] The positions D of j drones on the gridded map Ip The dimension is 1×J;

[0107] The location V of at most I vehicle cluster heads not connected to 5G base stations in the gridded map Ip The dimension is 1×I;

[0108] In this embodiment, the grids in the grid map are numbered from 0 to 9999, therefore the above B Ip D Ip V IpThe “location” in the table is the grid number where the 5G base station, drone, or vehicle cluster head is located, with a value ranging from 0 to 9999.

[0109] Within a decision cycle, the vehicle cluster heads that join are numbered by the server as 1, 2...I, corresponding to vector V. Ip The first to the ith elements, each with a value representing the map grid number where the corresponding vehicle cluster head was located when it issued the request, are exemplified in Table 2, which shows the vector V. Ip The correspondence between elements and cluster head numbers:

[0110] Table 1 Vector V Ip The correspondence between elements and cluster head numbers in the database.

[0111]

[0112] As shown in the table, when the number of currently connected vehicle cluster heads is less than I, it is padded with -1 to make V. Ip Items in the text.

[0113] Drone battery E j The dimension is 1×J, where the element value ranges from 0 to 1, and the closer to 1, the more energy is available.

[0114] UAV current mission status S j The dimension is 1×J, where the element value ranges from 0 to 1, where 0 means not executing a task and 1 means executing a task.

[0115] Therefore, the total dimension of the state space is 3×K+1×J+1×I, and the state space can be represented as:

[0116] State = {B Ip D Ip E j S j V Ip},

[0117] The network action space of the decision model has dimension J. The output value of each dimension is mapped to an integer between 0 and 1, representing the target vehicle cluster head number of the UAV, so as to obtain the position of the vehicle cluster head at any time. A value of 0 indicates that the UAV has not been assigned any vehicle cluster head. The action space can be represented as:

[0118] Action = {a0, a1, a2, ..., a...} J}, a j ∈[0, 1, 2, ..., I],

[0119] In this embodiment, the reward function is designed as follows:

[0120] Reward = kr D total -r p ,

[0121] Where k r =0.01 is a constant coefficient, k r D total The reward comes from system throughput. To reduce the occurrence of multiple drones serving a single cluster head, a penalty term r is added to the reward function. p When the output values ​​are a0, a1, a2..., a... J After mapping to integers 0 to 1, if there are duplicate numbers other than 0, then r... p =1, otherwise r p =0. As can be seen from the above reward function, the reward obtained during training is only related to the actual data throughput of the drone, and if multiple drones are assigned to the same cluster head, a penalty will be imposed. This ensures that in each decision of the trained model, each vehicle cluster head will only be assigned to one drone.

[0122] In this embodiment, the deep reinforcement learning neural network used by the model is configured to consist of two network models: a decision network and a target network. The initial settings of both network models are identical, each containing one input layer, two hidden layers, and one output layer. The first hidden layer has 256 neurons, the second hidden layer has 128 neurons, the number of neurons in the input layer is the same as the dimension of the state space, and the number of neurons in the output layer is the same as the dimension of the action space. In this embodiment, the number of training generations is set to episode = 200,000, and the neural network learning rate is set to 1. rt =0.0001, reward function discount factor γ = 0.99, and the capacity of the experience cache pool is set to b. capcity =200, the size of the empirical replay sample is b size =64. During training, the policy network is updated continuously first, and the target network is updated only after each training iteration when the experience buffer is full. The settings of other hyperparameters of the neural network are based on the literature Zhang T, Lei J, Liu Y, et al. Trajectory optimization for UAV emergency communication with limited user equipment energy: A safe-DQN approach[J].IEEE Transactions on Green Communications and Networking, 2021, 5(3): 1236-1247.

[0123] After defining the state space and action space of the decision-making model, we need to wait for the UAV trajectory optimization model to be trained before training the server-side decision-making model, because the server-side model training process needs to use the data throughput from the UAV to represent the quality of each decision.

[0124] Given the total bandwidth B of the drone d And the communication bandwidth of a single vehicle B i Therefore, the maximum number of vehicles that a drone can communicate with at the same time is:

[0125]

[0126] The transmission rate of the UAV and vehicle cluster head i at time t is expressed as:

[0127]

[0128] Where P c The communication power d for the UAV to provide communication services to a single vehicle cluster head i i Let n be the distance between vehicle cluster head i and UAV, n0 be the noise power spectral density, and α be the minimum communication radius of UAV.

[0129] Where L i (t) represents the path loss between vehicle cluster head i and the UAV, and we have:

[0130]

[0131] Where spl represents the spatial path loss. The probability of connecting at the Loss of Sight (LOS). The probability of NLoS (non-line-of-sight) connection, and η LoS With η NLoS The transitional path loss between Los and NLoS connections;

[0132] Based on the drone's altitude And drones can calculate the probability of line-of-sight connectivity:

[0133]

[0134] Where η a =12.08, η b =0.11 is a parameter related to the propagation environment. Where f c For the carrier frequency, v c It is the speed of light.

[0135] In this embodiment, the above parameter is set as follows: η LoS =1.6dB, η NLoS =23dB, n0=10 -17 W / Hz.

[0136] At time t, the distance d between the vehicle cluster head and the drone is calculated based on the drone's position and the vehicle cluster head's position. i If the distance is less than the minimum UAV communication radius α, then communication can be established with the UAV. In this embodiment, α is 300 meters. It is known that at time t, M (M≤N) c The vehicle cluster heads meet the communication range requirements and can connect with the UAV. To facilitate problem solving, the continuous time is discretized into N time intervals of length t. N The time slots, within the entire mission cycle T, have Let n = 0, 1, 2...N, and t0 be the start time of the mission. Assume that the transmission rate, speed, and other attributes of the UAV remain constant within a time slot, and take t = nt. N Given the value at time +t0, the total data throughput of a single drone within time slot n is:

[0137] R total (n)=R i (nt N +t0)t N .

[0138] Energy consumption of the UAV is one of the influencing factors in trajectory planning. This embodiment mainly considers three aspects of energy consumption: the energy consumption of the UAV moving at a fixed speed, the energy consumption of the UAV hovering at a certain position, and the energy consumption of communication between the UAV and the vehicle cluster head. This invention assumes that the UAV flies horizontally at a fixed speed throughout its flight. The thrust of a single UAV propeller can then be expressed as:

[0139]

[0140] Where, n r S represents the number of propellers, m represents the weight of the drone, and S represents the weight of the drone. FP The equivalent flat surface area of ​​the fuselage, g is the Earth's gravitational acceleration, ρ is the density of air, v is the speed of the drone, and V d It is a vector used to represent the velocity direction of the drone; based on the thrust of each drone propeller, we can obtain the propulsion power of the drone moving at velocity v as follows:

[0141]

[0142] Where δ is the local drag coefficient of the blade section, c TLet A be the thrust coefficient based on disk area, where A is the disk area of ​​each rotor, and c is the thrust coefficient based on disk area. s c is the rotor stability coefficient. f Incremental correction coefficient for induced power, d is the tilt angle, and d0 is the fuselage drag ratio for each rotor. In this embodiment, reference is made to Ding R, Gao F, Shen X S. 3D UAV trajectory design and frequency band allocation for energy-efficient and fair communication: A deep reinforcement learning approach[J]. IEEE Transactions on Wireless Communications, 2020, 19(12): 7796-7809. In this embodiment, the UAV-related parameter settings mentioned above are shown in the table below:

[0143] Table 2. UAV Relevant Parameter Settings

[0144]

[0145]

[0146] The drone moves at a speed v within a time slot, and the energy it consumes is t. N P move Similarly, the energy consumption when the drone is hovering can be derived as the drone hovering power P. hover The energy consumed by the drone hovering within the time slot is t. N P hover The communication power consumption of the drone is P. c Then the communication energy consumption of the UAV within the time slot is t. N P c The total energy at takeoff of the drone is represented as E. max Then the remaining energy state iteration of the drone can be expressed as:

[0147] E(n+1)=E(n)-(σt N P move +(1-σ)t N P hover +t N P c ),

[0148] Where σ represents the motion state of the UAV – motion or hovering, with a value of 0 or 1, where 1 represents motion and 0 represents hovering. In this embodiment, t NThe value is fixed at 0.2s;

[0149] The flight control of the drone adopts a fixed speed motion. Based on some drone design and simulation literature, this embodiment finds that there is an optimal flight speed that can minimize the drone's motion energy consumption. Therefore, in this embodiment, the drone's speed is fixed at 15m / s to improve energy utilization efficiency.

[0150] In summary, the online decision-making problem for drones, i.e., the drone trajectory optimization problem, can be transformed into an optimization problem that maximizes the drone's data throughput while satisfying the drone's energy constraints and obstacle avoidance requirements.

[0151] P2:

[0152] Furthermore, the following constraints must also be met:

[0153] C1: E T >0

[0154] C2:

[0155] C3: CL = 0

[0156] C4:x min <x<x max

[0157] C5:y min <y<y max

[0158] C6:z min <z<z max ;

[0159] E in C1 T This represents the energy of the drones at the end of the mission; this constraint determines that all drones have enough energy to return to the base station.

[0160] D in C2 pT B represents the three-dimensional position of the UAV at the end of the mission. P This represents the set of three-dimensional locations of 5G base stations, and this constraint determines that all drones can take off from and land at the base stations.

[0161] In C3, CL represents the number of collisions. This constraint determines that the drone should avoid colliding with obstacles or other drones during flight.

[0162] In C4-C6, x, y, and z represent the three-dimensional coordinates of the UAV. These three constraints determine that the UAV's flight altitude should be greater than the minimum altitude and less than the maximum altitude, and it should not fly out of the mission area.

[0163] The UAV trajectory optimization model in this embodiment is used to solve the above problem. This model is also a DQN model based on reinforcement learning. This model transforms the UAV path optimization problem into considering the three-dimensional position V of the vehicle cluster head. p UAV energy E(n), UAV three-dimensional position D p Obstacle information sensed by lidar O s Base station location B Ip The optimization problem of maximizing drone data throughput under factors such as [list of factors].

[0164] The neural network of this model takes the factors mentioned above as input and outputs the flight maneuver that the drone should perform in the next moment. For simplicity, in this embodiment, as... Figure 3 As shown, O s A fixed 20-dimensional list is used to represent the obstacle distribution around the drone; specifically, ten directional angle intervals are evenly distributed on the horizontal plane and ten directional angle intervals are evenly distributed on the vertical plane, centered on the drone. The width of each directional angle interval is 360° / 10 = 36°, and the detection distance is 5m. A value of 1 indicates that there is an obstacle in the interval, and a value of 0 indicates that there is no obstacle. Therefore, the state space of the model is:

[0165] S={V p E(n), D p O s B Ip},

[0166] The state space has a dimension of 3+1+3+20+k, where k is the number of base stations;

[0167] The action space of the model's output is:

[0168] A={left, right, forward, backward, up, down, hover},

[0169] It is easy to see that in the action space, different variables represent different flight actions. When left is 1, it means to go left; when right is 1, it means to go right; when forward is 1, it means to move forward; when backward is 1, it means to move backward; when up is 1, it means to rise; when down is 1, it means to fall; and when hover is 1, it means to hover. The action space output points to the flight action that the drone will perform in the next moment. Therefore, the values ​​of the variables are 0 and 1, and only one variable becomes 1 each time.

[0170] The reward function for this model is:

[0171]

[0172] Where k th R total (n) represents the reward for the data throughput of the drone in time slot n, k th =0.00001 is a constant weighting coefficient. To shape rewards for drones in relation to target points and energy, and to prevent reward sparsity during training, k rd =0.5 is a constant weighting coefficient. Used to prevent the denominator from being zero. d represents the energy consumed within a time slot. dis r represents the reduction in distance between the UAV and the destination within the current time slot. c The reward for a drone landing at a base station, r ad The penalty for a drone collision is represented in this embodiment, where the hyperparameter settings of the neural network are consistent with those of the scheduling decision neural network in the first layer.

[0173] It is worth noting that in practical applications, the input data for this model will be pre-set differently depending on the drone's operating state. Specifically:

[0174] When the drone is performing a mission, force setting B Ip All values ​​in the data are -1; that is, during the drone's mission, the base station location B... Ip It does not affect the flight path planning decisions of the drone;

[0175] When the drone is en route back to the base station, the input V is forcibly set. p The value is (0, -1, 0), meaning that after the UAV completes its mission, the three-dimensional position V of the target cluster head is... p No longer affecting the drone's flight path planning decisions, the drone, based on its current location and battery level, decides to return to a certain base station's flight path under constraints, while simultaneously performing obstacle avoidance in real time.

[0176] In this embodiment, when the UAV receives the command to end the auxiliary data transmission task, it will switch the decision model setting from "in the process of executing the task" to "on the way back to the base station". The server can issue the task end command according to the status of the target vehicle cluster head. For example, after the target vehicle cluster head can intervene in the 5G base station, the server issues the task end command to the UAV serving the target vehicle cluster head.

[0177] Finally, the series of flight maneuver decisions made by the drone form the drone trajectory, represented by the action set X.

[0178] To reduce the costs associated with frequent drone crashes during physical training, this embodiment proposes a cost-effective training method. First, an elevation map of the deployment area is downloaded from Google Earth, followed by satellite maps and vegetation data for that area. The elevation data is imported into a Unity simulation platform to generate the terrain of the area, and then the distribution of buildings and vegetation is adjusted using the satellite maps. This creates a near-realistic training simulation environment. The constructed DQN drone decision-making model is then trained within this environment. The training objective for the drone decision-making model is to maximize the drone's data throughput. After the drone's DQN model reaches a certain accuracy, a cloud-based decision-making model is trained, with the amount of data collected by the drone being a crucial factor fed back to the cloud model. After both layers of DQN model training are completed, they are deployed on a cloud server and on individual drones, respectively.

[0179] After the offline model training is completed, the model is deployed on the drone and the server, and then the drone is randomly deployed to 5G base stations within the mission range.

[0180] Once deployed, the cloud server constantly monitors feedback from vehicles and 5G base stations within its range. If a vehicle is outside the 5G base station's communication range, it requests data directly from the cloud via the 4G cellular network, informing the cloud of its location. If it is within the 5G base station's communication range, it requests data from the 5G base station. All 5G base stations upload their data request information to the cloud server at a fixed frequency. They also inform the cloud decision server whether the base station currently hosts a drone. If so, they upload the drone's status information. The determination of whether a base station hosts a drone is based on whether a drone is parked and charging at that base station.

[0181] The cloud server parses information from vehicles or 5G base stations and determines whether drone assistance is needed. If the data request comes from a 5G base station, the data is sent to the corresponding base station without scheduling the drone, while the server continues to monitor data from vehicles and base stations.

[0182] For data requests sent directly from vehicles to the cloud via 4G cellular networks, the cloud server uses pre-trained algorithms to make decisions based on the distribution of 5G base stations, drone locations, drone energy status, and vehicle cluster distribution.

[0183] The server uses a cloud-based drone scheduling model to assign target cluster heads to each drone and sends the target cluster head number to the drone. The drone then uses the number to obtain the real-time location of the target cluster head.

[0184] Once the drone reaches the target area, it determines whether the Euclidean distance to the vehicle cluster head meets the communication requirements. If so, it establishes communication with the corresponding vehicle cluster head. Using an onboard micro 5G base station, it establishes contact with the cluster heads within communication range. Then, the data exchanged between the vehicles is transmitted to a macro base station via a CPE, and the macro base station connects to the 5G core network, thus realizing vehicle-to-Internet data transmission.

[0185] As described in the aforementioned drone trajectory optimization model, in addition to data interaction with the vehicle, the drone also utilizes its onboard sensors to detect the surrounding environment. First, it continuously obtains its own position using its onboard positioning module. Then, it uses its onboard 64-line LiDAR to continuously scan the surrounding 3D environment to provide information on the distribution of obstacles around the drone. The drone uses the interaction information with the vehicle and the surrounding environment detection information to form decision-making conditions for its next action. These next actions include forward, backward, left, right, up, and down movements, as well as hovering.

[0186] After completing its mission or when its battery is sufficient to complete the mission, the drone selects a 5G base station to dock and recharge, based on its location and battery level. It then uploads the mission completion status to the server and awaits the next dispatch command. When the drone needs to return to a base station for charging, it switches its current target point to the nearest base station and continues its autonomous obstacle avoidance flight to the destination base station.

[0187] If the uploaded task is incomplete, the cloud server will make a decision and dispatch the drone to complete the remaining task based on the request information, vehicle location, other drone locations, and battery level.

[0188] If the drone has no data transmission mission, it should park and charge at the 5G base station, waiting for the next scheduling mission. At the same time, the 5G base station will also periodically upload its own and the drone's status information to the cloud decision server.

Claims

1. A method for unmanned aerial vehicle (UAV)-assisted vehicle-to-everything (V2X) data transmission, characterized in that, include: Step 1: Randomly deploy multiple drones at 5G base stations within the mission area. Each drone is equipped with a 5G micro base station and a CPE device. Step 2: The server receives data requests from vehicle cluster heads within the task area. If the vehicle cluster head is connected to a 5G base station, it will request data through the 5G base station. If the vehicle fails to connect to the 5G base station, it requests data from the server via the cellular network; Step 3: The server integrates data request information from all 5G base stations and vehicles, uses the cloud-based drone scheduling model deployed on the server to make scheduling decisions based on the status information of base stations, vehicles, and drones, and sends the target vehicle cluster head ID corresponding to the selected drone to the drone. The cloud-based drone scheduling model is a pre-trained decision model aimed at maximizing system data throughput, which includes the data throughput of 5G base stations and the data throughput of drones. Step 4: After each UAV receives its target vehicle cluster head ID, it accesses the real-time location of the target vehicle cluster head from the server based on the ID, and uses the flight path planning model deployed on the UAV itself to plan the UAV's flight path based on its own status information and the real-time location of the target vehicle cluster head. The unmanned flight route planning model is a pre-trained decision model aimed at maximizing the data throughput of the unmanned aerial vehicle. The unmanned flight route planning model adopts a DQN model, and the state space of the DQN model for unmanned flight route planning is: S={V p ,E(n),D p ,O s ,B Ip }, Among them, V p This indicates that the current target vehicle cluster head has a 3D position dimension of 3; E(n) represents the drone's energy at the current moment, with dimension 1, n = 0, 1, 2...N, indicating that the energy is expressed in terms of time t. N The discrete time is the time slot length; D p This indicates the current three-dimensional position of the drone, with a dimension of 3; O s O represents the obstacle information currently sensed by the lidar. s A fixed 20-dimensional list is used to represent the distribution of obstacles around the drone, and its configuration strategy is as follows: Within a certain detection distance centered on the drone, ten directional angle intervals evenly distributed on the horizontal plane and ten directional angle intervals evenly distributed on the vertical plane correspond to O respectively. s The 20 dimensions, each directional angle interval has a width of 360° / 10 = 36°. A value of 1 indicates that there is an obstacle in that interval, and a value of 0 indicates that there is no obstacle. The action space of the model's output is: A={left,right,forward,backward,up,down,hover}, When left is 1, it means to move left; when right is 1, it means to move right; when forward is 1, it means to move forward; when backward is 1, it means to move backward; when up is 1, it means to rise; when down is 1, it means to fall; when hover is 1, it means to hover. Only one variable will change to 1 at any given time. The reward function for training the DQN model used for unmanned flight path planning is: Where, k th R total (n) represents the reward for the data throughput of the drone in time slot n, k th =0.00001 is a constant weighting coefficient. To shape rewards related to the drone's target point and energy, and to prevent reward sparsity during training, k rd =0.5 is a constant weighting coefficient. Used to prevent the denominator from being zero. d represents the energy consumed within a time slot. dis r represents the reduction in distance between the UAV and the destination within the current time slot. c The reward for a drone landing at a base station, r ad This represents the penalty for a drone collision; The total data throughput of the UAV within time slot n is: R total (n)=R i (nt N +t0)t N ; Where t0 is the mission initiation time, and the transmission rate between the UAV and vehicle cluster head i at time t is expressed as: Among them, P c The communication power d for the UAV to provide communication services to a single vehicle cluster head i i Let n be the distance between vehicle cluster head i and UAV, n0 be the noise power spectral density, and α be the minimum communication radius of UAV. Where L i (t) represents the path loss between vehicle cluster head i and the UAV, where: Where spl represents the spatial path loss. The probability of line-of-sight connection. For non-line-of-sight connections, and η LoS With η NLoS The excessive path loss for connecting Los and NLoS; Based on the drone's altitude And drones can calculate the probability of line-of-sight connectivity: Where η a η b For parameters related to the propagation environment, Where f c For the carrier frequency, v c The speed of light; The model should also meet the following constraints during training: C1:E T >0 C2: C3:CL=0 C4:x min <x<x max C5:y min <y<y max C6:z min <z<z max ; E in C1 T This indicates the drone's energy level at the end of the mission; D in C2 PT B represents the three-dimensional position of the UAV at the end of the mission. P A set representing the three-dimensional locations of base stations; In C3, CL represents the number of collisions; In C4-C6, x, y, and z represent the three-dimensional coordinates of the UAV; During training and decision-making, the input data for this model is pre-set according to the following strategy: When the drone is performing a mission, force setting B Ip All values ​​in the table are -1; When the drone is en route back to the base station, the input V is forcibly set. p (0, -1, 0); Step 5: After the drone arrives at the target vehicle cluster, it uses its onboard 5G micro base station and CPE equipment to establish a wireless transmission network between the vehicle and the macro base station and transmit data. The task area is divided into multiple map grids, each of which is uniquely numbered. The state space of the DQN model used for cloud-based drone scheduling is: State={B Ip ,D Ip ,E j ,S j ,V Ip }, Among them, B Ip Let K be the locations of 5G base stations in a gridded map, with a dimension of 1×K; D Ip Let J be the positions of J drones in a gridded map, with dimension 1×J; V Ip The location of at most I vehicle cluster heads that are not connected to a 5G base station in the gridded map, with a dimension of 1×1; The B Ip D Ip V Ip The location in the text is the grid number where the 5G base station, drone, or vehicle cluster head is located; Vehicle clusters that fail to access a 5G base station within a decision-making cycle are numbered by the server as 1, 2...I, corresponding to vector V. Ip The first to the ith element, each value being the map grid number where the corresponding vehicle cluster head was located when it issued the request. If the number of vehicle cluster heads that have failed to connect to the 5G base station is less than ith, then -1 is used to fill the gap to ith. Ip Items in; E j The value represents the power of the drone, with a dimension of 1×J, where the element value ranges from 0 to 1, and the closer to 1, the more energy is available; UAV current mission status S j The dimension is 1×J, where the element value ranges from 0 to 1, where 0 means not executing a task and 1 means executing a task. The DQN model used for cloud-based drone scheduling has a network action space dimension of J. The output value of each dimension is mapped to an integer between 0 and 1, representing the target vehicle cluster head number of the drone, so as to obtain the position of the vehicle cluster head at any time. A value of 0 indicates that the drone has not been assigned any vehicle cluster head. The action space can be represented as follows: Action={a0,a1,a2...,a J },a j ∈[0,1,2,...,I]; The reward function used to train the DQN model for cloud-based drone scheduling is: Reward=k r D total -r p , Among them, D total The data throughput for data transmission using drone assistance is expressed as: Where R j,total This represents the total data transmission volume of drone j; Where k r =0.01 is a constant coefficient, and the penalty term r p When the output action space contains a0, a1, a2..., a J After mapping to integers 0 to 1, if there are duplicate numbers other than 0, then r... p =1, otherwise r p =0.

2. The method according to claim 1, characterized in that, It also includes step 6: After the drone-assisted transmission mission is completed, it flies back to any 5G base station in the mission area to park and charge. The unmanned flight route planning model is also used to plan the route for the drone to fly back to a certain 5G base station based on its own status information and the location of the 5G base station after the drone has completed the task of assisting in transmission.

3. The method according to claim 1, characterized in that, The cloud-based drone scheduling model adopts a DQN model.

4. The method according to claim 1, wherein the UAV is equipped with a lidar for sensing obstacle information, used for UAV scheduling model, and for planning the flight route of the UAV based on the obstacle information; The model is trained under the constraint of not colliding with obstacles.

5. A drone-assisted vehicle-to-everything (V2X) data transmission system, characterized in that, This includes 5G base stations, cloud servers, and drones; The drone is equipped with a positioning module and a wireless transmission module; The 5G base station, cloud server, and drone are interconnected. The system assists in the data transmission of the vehicle network according to any one of the methods described in claims 1-4.

Images