Traffic flow prediction-based unmanned aerial vehicle-assisted Internet of Vehicles trajectory planning method and system

Abstract

The invention discloses an unmanned aerial vehicle assisted Internet of Vehicles trajectory planning method and system based on traffic flow prediction, and the method comprises the steps: S1, employing multi-source historical traffic data collected by a distributed sensor network as input, and building a traffic flow space-time prediction model based on a deep learning algorithm; s2, a vehicle networking model composed of a plurality of vehicles and a plurality of unmanned aerial vehicles is constructed, the unmanned aerial vehicles fly at a fixed height and serve as air base stations to provide services for ground vehicles, and only one unmanned aerial vehicle provides services for each vehicle; and S3, constructing a multi-unmanned aerial vehicle assisted Internet of Vehicles communication system model, clustering ground vehicles by using a clustering algorithm in combination with traffic flow prediction data, and further jointly optimizing trajectory planning and resource allocation strategies of the unmanned aerial vehicles by taking a clustering center as a pre-deployment position of the unmanned aerial vehicles for each vehicle cluster. The corresponding system comprises the sensor network, the cloud platform and the unmanned aerial vehicle cluster carrying the base station, and the communication quality and the energy efficiency of the Internet of Vehicles can be effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of wireless communication technology, and particularly relates to a UAV-assisted V2X trajectory planning method and system based on traffic flow prediction. BACKGROUND

[0002] With the rapid development of 5G communication and intelligent networked vehicles, the demand for high reliability and low latency communication in V2X is increasing. However, traditional ground base stations are limited by fixed deployment and coverage, making it difficult to meet the high-quality communication needs of high-speed moving vehicles, especially in urban congested sections or remote areas. UAVs, with their high mobility, flexible deployment and line-of-sight transmission advantages, have become an important supplement to V2X communication, effectively improving network coverage and communication quality.

[0003] Currently, the research on UAV-assisted V2X mainly focuses on trajectory optimization and resource allocation. In terms of trajectory optimization, existing methods usually plan paths based on static or short-term traffic information, which is difficult to adapt to dynamically changing traffic flow. For example, some studies use reinforcement learning to optimize UAV trajectories, but due to the high dynamics of the V2X environment, traditional algorithms are prone to local optimization, resulting in decreased communication efficiency. In terms of resource allocation, existing solutions assume that vehicles are uniformly distributed, failing to fully consider the spatiotemporal characteristics of traffic flow, resulting in low utilization of spectrum and power resources.

[0004] In addition, the existing UAV-assisted V2X system has the following technical problems:

[0005] Lack of accurate traffic flow prediction: Traditional methods rely on real-time sensing data and are difficult to predict traffic congestion or sudden situations in advance, resulting in delayed UAV scheduling and affecting communication service quality.

[0006] Low efficiency of vehicle-UAV matching: In high-density traffic scenarios, the association optimization between UAVs and vehicles is complex, and existing clustering algorithms such as K-means cannot accurately reflect the spatial distribution characteristics of vehicles, resulting in suboptimal initial deployment locations.

[0007] Energy consumption and communication rate are difficult to balance: The joint optimization problem of UAV flight energy consumption and communication power is a non-convex optimization, and existing heuristic algorithms have slow convergence speed, making it difficult to meet real-time requirements.

[0008] Therefore, it is necessary to design a UAV-assisted V2X trajectory planning method and system based on traffic flow prediction to solve the above problems. SUMMARY

[0009] The UAV-assisted V2X trajectory planning method and system based on traffic flow prediction are proposed to solve at least one of the technical problems in the prior art.

[0010] To achieve the above object, the application adopts the following technical solutions:

[0011] The unmanned aerial vehicle assisted vehicle networking trajectory planning method based on traffic flow prediction comprises the following steps:

[0012] S1: Multi-source historical traffic data collected by a distributed sensor network is used as input, and a traffic flow spatio-temporal prediction model is established based on a deep learning algorithm; the multi-source historical traffic data is uploaded to a cloud platform or a centralized data processing node in real time or periodically through a wireless communication network, and after data cleaning, integration and analysis, a prediction result is generated;

[0013] S2: A vehicle networking model composed of multiple vehicles and multiple unmanned aerial vehicles is constructed, M unmanned aerial vehicles are deployed, and U vehicles are randomly distributed in the region, wherein the unmanned aerial vehicle set is represented as M, and Each unmanned aerial vehicle autonomously decides in each time slot t to optimize the flight path and select the best service object;

[0014] S3: An unmanned aerial vehicle trajectory optimization method based on traffic flow prediction data is established, specifically including: combining traffic flow prediction data and using a clustering algorithm to cluster ground vehicles, taking the cluster center as the pre-deployment position of the unmanned aerial vehicle for each vehicle cluster, and further jointly optimizing the trajectory planning and resource allocation strategy of the unmanned aerial vehicle; the MATD3 algorithm is used to optimize the three-dimensional trajectory planning of the unmanned aerial vehicle to determine the flight path and transmission power of the unmanned aerial vehicle.

[0015] Preferably, the entire data processing in S1 includes data collection, preprocessing, model training and prediction model evaluation;

[0016] In the preprocessing stage, the historical data is systematically evaluated, the distribution characteristics of missing data and its potential impact are analyzed, and the mean filling method is used to repair the missing data. After filling the missing values, a variety of data transformation strategies are applied, and Z-score normalization is used for standardization processing.

[0017] Preferably, after completing the preprocessing of the data in S1, based on the time sequence characteristics and complex relationships of the data, LSTM, GRU, CNN and multilayer perception network are selected to construct the prediction model, the data set is divided into training set and test set, the model is trained and optimized, and the generalization ability of the model is evaluated on the test set.

[0018] Preferably, the vehicle networking model in S2 satisfies the following constraint conditions: each vehicle is served by only one unmanned aerial vehicle; the unmanned aerial vehicle makes a trajectory planning decision in each decision time slot; and the communication coverage area of the unmanned aerial vehicle is a predefined geographical range.

[0019] Preferably, S3 specifically includes:

[0020] S31: clustering the ground vehicles using a Gaussian mixture model to determine initial deployment locations for the UAVs;

[0021] S32: establishing a three-dimensional coordinate model to calculate the distance between the UAVs and the vehicles, the line-of-sight transmission probability, and the channel gain;

[0022] S33: constructing a non-convex optimization problem including the trajectory variables and the transmission power variables of the UAVs;

[0023] S34: jointly optimizing the trajectory planning and the power allocation of the UAVs using a multi-agent reinforcement learning algorithm.

[0024] Preferably, the multi-agent reinforcement learning algorithm in S34 uses a MATD3 algorithm, which includes: configuring an independent Actor network and a double Critic network for each UAV; the Critic network accesses the state and action information of all UAVs during the training process; a delayed policy update mechanism and an action noise clipping technique are used; the reward function design takes into account the maximization of the communication rate, the minimization of the energy consumption, and the satisfaction of the constraint conditions.

[0025] Preferably, the state space of the MATD3 algorithm is defined as the coordinate position of the UAVs in the time slot, the action space includes the three-dimensional motion vector of the UAVs and the transmission power setting, and the design of the reward function includes: the positive reward is based on the average communication rate of the system; the negative punishment is based on the total energy consumption of the UAVs; and an additional penalty term is used to constrain the violation of the conditions.

[0026] Preferably, the EM algorithm is used in S31 to estimate the parameters of the Gaussian mixture model, which includes: initializing the number of mixture components and the parameters of each component; iteratively performing the expectation step to calculate the posterior probability, and the maximization step to update the mean, covariance, and mixing coefficient; and determining the vehicle cluster division and the center position according to the converged model parameters.

[0027] An unmanned aerial vehicle (UAV) assisted vehicle networking system, comprising: a distributed sensor network for collecting multi-source historical traffic data;

[0028] A cloud platform for implementing the step S1 of the method according to any one of claims 1-8;

[0029] A plurality of UAVs configured with communication base stations for executing the optimized trajectory generated by the method according to any one of claims 1-8;

[0030] A vehicle terminal device for establishing a communication connection with the UAVs.

[0031] Compared with the prior art, the UAV assisted vehicle networking trajectory planning method and system based on traffic flow prediction provided by the present application have the following beneficial effects:

[0032] 1. The application collects multi-source historical traffic data through a distributed sensor network and constructs a high-precision spatiotemporal prediction model based on deep learning, which can predict the dynamic changes of traffic flow in advance, so that the UAV can actively adjust the flight trajectory to avoid communication delay caused by traffic congestion or sudden conditions, and significantly improve the service quality of the Internet of Vehicles.

[0033] 2. The application adopts a Gaussian mixture model combined with an EM algorithm to more accurately identify vehicle clusters and use the cluster center as the optimal pre-deployment position of the UAV, reducing the search space of reinforcement learning, improving optimization efficiency, and reducing the additional energy consumption of the UAV caused by frequent position adjustment.

[0034] 3. The application adopts the MATD3 algorithm, which effectively avoids the Q-value overestimation problem of traditional reinforcement learning through a double Critic network and a delay policy update mechanism, so that the UAV can quickly converge in a complex dynamic environment and realize joint optimization of trajectory and resources. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 A flowchart of the UAV-assisted trajectory planning method and system for the Internet of Vehicles based on traffic flow prediction according to the application;

[0036] Figure 2 A traffic flow prediction system model diagram of the UAV-assisted trajectory planning method and system for the Internet of Vehicles based on traffic flow prediction according to the application;

[0037] Figure 3 A traffic flow prediction flowchart of the UAV-assisted trajectory planning method and system for the Internet of Vehicles based on traffic flow prediction according to the application;

[0038] Figure 4 An Internet of Vehicles system model diagram of the UAV-assisted trajectory planning method and system for the Internet of Vehicles based on traffic flow prediction according to the application;

[0039] Figure 5 A MATD3 algorithm framework diagram of the UAV-assisted trajectory planning method and system for the Internet of Vehicles based on traffic flow prediction according to the application;

[0040] Figure 6 A clustering algorithm comparison result diagram of the UAV-assisted trajectory planning method and system for the Internet of Vehicles based on traffic flow prediction according to the application;

[0041] Figure 7 A vehicle clustering result diagram of the GMM clustering algorithm of the UAV-assisted trajectory planning method and system for the Internet of Vehicles based on traffic flow prediction according to the application;

[0042] Figure 8The predicted vehicle distribution map of the unmanned aerial vehicle assisted vehicle networking trajectory planning method and system based on traffic flow prediction according to the present application;

[0043] Figure 9 The flight trajectory map of multiple unmanned aerial vehicles of the unmanned aerial vehicle assisted vehicle networking trajectory planning method and system based on traffic flow prediction according to the present application;

[0044] Figure 10 The change graph of the average reward value with the increase of the number of rounds of the unmanned aerial vehicle assisted vehicle networking trajectory planning method and system based on traffic flow prediction according to the present application;

[0045] Figure 11 The average reward performance comparison graph of the MATD3 algorithm of different learning rates of the unmanned aerial vehicle assisted vehicle networking trajectory planning method and system based on traffic flow prediction according to the present application;

[0046] Figure 12 The average communication rate change graph of vehicles under different transmission powers of the unmanned aerial vehicle assisted vehicle networking trajectory planning method and system based on traffic flow prediction according to the present application;

[0047] Figure 13 The system energy efficiency graph under different vehicle quantities of the unmanned aerial vehicle assisted vehicle networking trajectory planning method and system based on traffic flow prediction according to the present application. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments.

[0049] In the description of the present application, it should be understood that the terms "upper", "lower", "front", "back", "left", "right", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, therefore it cannot be understood as a limitation of the present application.

[0050] Please refer to the drawings shown in the present application Figures 1-13 The present embodiment provides an unmanned aerial vehicle assisted vehicle networking trajectory planning method based on traffic flow prediction, which comprises the following steps:

[0051] S1: Using multi-source historical traffic data collected by distributed sensor networks as input, a spatiotemporal traffic flow prediction model is established based on deep learning algorithm; the multi-source historical traffic data is uploaded to the cloud platform or centralized data processing node in real time or periodically through wireless communication network, and after data cleaning, integration and analysis, the prediction result is generated;

[0052] S2: A vehicle networking model composed of multiple vehicles and a single unmanned aerial vehicle is constructed, containing U vehicles, and the unmanned aerial vehicle flies at a fixed height H to provide services for ground vehicles as an air base station, and the vehicle set is represented as U={1, 2,..., U}, in addition, each vehicle is only served by one unmanned aerial vehicle, and the unmanned aerial vehicle makes trajectory planning in each decision time slot;

[0053] S3: An unmanned aerial vehicle trajectory optimization method based on traffic flow prediction data is established, specifically including: determining the initial deployment position of the unmanned aerial vehicle based on the prediction result using the mean center method; establishing an optimization model with the goal of maximizing the network average rate and minimizing the energy consumption; using DDPG algorithm to optimize the three-dimensional trajectory planning of the unmanned aerial vehicle to determine the optimal hovering position and flight path.

[0054] In this embodiment, in the unmanned aerial vehicle assisted vehicle networking, the unmanned aerial vehicle needs to provide real-time communication services for ground vehicles, and the vehicle flow presents dynamic change characteristics affected by time factors and sudden events such as traffic accidents, road construction and bad weather, the development of artificial intelligence technology provides a new solution for vehicle networking flow prediction and unmanned aerial vehicle trajectory optimization. Through the deployment of sensor networks, visual collection equipment and vehicle-mounted terminals in road infrastructure, real-time monitoring of traffic parameters such as vehicle position, speed and density can be realized. Based on these data, combined with machine learning or deep learning algorithm, a high-precision short-term traffic flow prediction model can be constructed. In the aspect of prediction model construction, this paper uses multi-source historical traffic data collected by distributed sensor networks as input, and establishes a spatiotemporal traffic flow prediction model based on deep learning algorithm. This model can effectively capture the time variation characteristics and spatial correlation of traffic flow, thereby accurately predicting the flow fluctuation under normal and sudden conditions, and providing the basis for dynamic adjustment of unmanned aerial vehicle trajectory optimization. In addition, in order to improve the quality of service of vehicle networking, based on the high-precision traffic flow prediction result, an unmanned aerial vehicle assisted communication system framework is constructed, and an unmanned aerial vehicle trajectory optimization method based on traffic flow prediction data is proposed. Specifically, to reduce the computational complexity of the optimization problem, GMM algorithm is used to cluster ground vehicles, solve the matching between unmanned aerial vehicles and vehicles, determine the initial deployment position of unmanned aerial vehicles, and jointly optimize the trajectory planning and resource allocation of unmanned aerial vehicles based on MATD3 algorithm. This method has significant improvement in system average rate and energy efficiency.

[0055] As Figure 2As shown, in this embodiment, the multi-source historical traffic data is uploaded to the cloud platform or centralized data processing node in real time or periodically through the wireless communication network, and the prediction results are generated after data cleaning, integration and analysis. Among them, part of the prediction data can be directly applied to the traffic management system to support intelligent traffic applications such as dynamic signal light control and road condition monitoring; another part of the data is used for decision-making basis for unmanned aerial vehicle assisted Internet of Vehicles through cloud data modeling and calculation to optimize the flight trajectory and resource allocation strategy of unmanned aerial vehicles.

[0056] In traffic data analysis, the completeness and quality of data directly affect the prediction accuracy of the model. In the actual data collection process, affected by factors such as bad weather, communication interruption, equipment failure or collection error, data loss or sparsity may occur in some areas. Such loss not only weakens the representation ability of key features, but also may reduce the generalization performance of the model. Therefore, in the data preprocessing stage, the historical data needs to be systematically evaluated, the distribution characteristics of the missing data and its potential impact are analyzed, and a reasonable missing value filling method is used for repair.

[0057] Missing value filling is a key step to improve data integrity, and its core goal is to reduce the interference of incomplete data on subsequent analysis and modeling by reasonably inferring and replacing missing values. An effective filling strategy can restore the statistical properties of the data set and enhance the robustness of the model. The commonly used filling methods include global filling based on statistical quantities (such as minimum value, maximum value, mean value or median value). In addition, the spatio-temporal correlation between data (such as data of adjacent time periods or adjacent road segments) can be used to make more accurate missing value estimation to improve the rationality of the filling results.

[0058] In a specific embodiment, as shown in Figure 3 In S1, the entire data processing includes data collection, preprocessing, model training and prediction model evaluation; in the data analysis and modeling process, missing data may affect the stability of model training and the accuracy of prediction. Therefore, reasonably and efficiently filling the missing data is an important and indispensable part of the data preprocessing stage. The present application uses the mean filling method to reduce the adverse effects of data incompleteness on model performance and improve data usability.

[0059] In the preprocessing stage, the historical data is systematically evaluated, the distribution characteristics of the missing data and its potential impact are analyzed, and the mean filling method is used to repair the missing data. To improve data quality and optimize analysis results, after filling the missing values, a variety of data transformation strategies are further applied to optimize data structure and enhance feature expression ability. In order to eliminate the numerical difference between different dimensions and improve data comparability, Z-score normalization is used for standardization processing, and the calculation formula is as follows:

[0060]

[0061] where μ is the mean, σ is the standard deviation, X normalized is the normalized data. Through this normalization method, the data distribution can be ensured to be balanced, avoiding unnecessary bias of the model caused by some feature values being too large or too small, and improving the stability and prediction accuracy of data analysis and model

[0062] In one specific embodiment, as shown in Figure 3 S1, after completing the data preprocessing, based on the time series characteristics and complex relationship of the data, LSTM, GRU, CNN and multi-layer perceptron network are selected to build prediction models, respectively. These models have unique advantages in time series dependence modeling, local feature extraction and nonlinear mapping, and can improve prediction accuracy from different angles. Subsequently, the data set is divided into training set and test set, the model is trained and optimized, and the generalization ability of the model is evaluated on the test set to ensure its stability and reliability in actual application.

[0063] In one specific embodiment, as shown in Figure 3 S1, after completing the model training, different methods are selected to judge the prediction model accuracy, and four evaluation indexes including mean absolute error, mean absolute percentage error, determination coefficient and root mean square error are used for evaluation.

[0064] The mean absolute error is used to measure the closeness between the prediction results and the true data, which is the average of all sample absolute errors, and can intuitively reflect the accuracy of model prediction. The smaller the value of mean absolute error, the better the fitting effect. The calculation method is as follows:

[0065]

[0066] where y i represents the true value, is the predicted value, and n represents the sample number.

[0067] The mean absolute percentage error is an index for measuring relative error, which reflects the deviation of the model by calculating the percentage of the prediction error and the true value. Compared with the mean absolute error, the mean absolute percentage error is more suitable for measuring the case where the data fluctuates greatly. The calculation method is as follows:

[0068]

[0069] The determination coefficient is an important index for measuring the fitting effect of the regression model, and is used to explain a measurement value of the fitting degree of the predicted data to the observed data, and the value range is between [0, 1], the closer the determination coefficient value is to 1, the higher the fitting degree of the model to the data; on the contrary, if the determination coefficient value is small, the fitting degree of the data is lower, and the specific calculation is as follows:

[0070]

[0071] Among them The average value of the true value is represented by

[0072] The root mean square error is used to measure the deviation between the predicted value and the true value, and it is sensitive to extreme errors in the data, so it can better reflect the stability and accuracy of the model prediction. The smaller the root mean square error is, the higher the prediction accuracy of the model is, and the calculation formula is as follows:

[0073]

[0074] In a specific embodiment, as shown in Figure 4 In S3, the initial deployment position of the unmanned aerial vehicle is determined based on the prediction result by using the mean center method; and an optimization model is established to maximize the network average rate and minimize the energy consumption, and the specific is: after establishing the vehicle networking model, a three-dimensional coordinate model is constructed, and the horizontal coordinate of the ground vehicle is represented as The horizontal coordinate of the unmanned aerial vehicle at time t is The distance from the unmanned aerial vehicle to the vehicle u at time slot t is represented as:

[0075]

[0076] Considering the downlink transmission scenario, the data transmission from the unmanned aerial vehicle to the vehicle is constructed as a probabilistic line-of-sight model, and it is assumed that the ground vehicle adopts the orthogonal frequency division multiple access mechanism to realize multiple access, that is, each vehicle exclusively occupies an orthogonal frequency division multiple access resource block for communication in any time slot, and the probability of line-of-sight transmission between the unmanned aerial vehicle and the vehicle u in time slot t is represented as The probability is approximated by the improved Sigmoid function as follows:

[0077]

[0078] Among them, x and y are parameters of the Sigmoid function, and u The angle of elevation from the unmanned aerial vehicle to the vehicle u is represented by

[0079]

[0080] The channel power gain between the unmanned aerial vehicle and the vehicle u is modeled according to the line-of-sight and non-line-of-sight conditions, and is represented as and The channel gain for UAV line-of-sight transmission with vehicle u in time slot t is expressed as:

[0081]

[0082] The channel gain expression for non-line-of-sight transmission is also obtained as follows:

[0083]

[0084] in, and Let represent the small-scale fast fading components of the UAV and vehicle u under time slot t for line-of-sight and non-line-of-sight transmission, respectively, and let λ be the mean. LoS and λ NLoS The exponential distribution, and Let ξ represent the log-normal shadowing fading random variable of UAV and vehicle u under line-of-sight and non-line-of-sight transmission in time slot t, with standard deviation ξ, A being the path loss constant, and d being the path loss constant. u Let κ be the distance from the drone to the vehicle u, and κ be the path loss exponent.

[0085] To simplify the problem, assume that the channel gain between the UAV and the vehicle remains constant within a time slot. Therefore, the total channel gain h of the transmission between the UAV and vehicle u in time slot t is... u [t] is represented as:

[0086]

[0087] The signal-to-noise ratio (SNR) of the drone and vehicle u within time slot t is expressed as:

[0088]

[0089] Where p[t] represents the transmit power of the UAV in time slot t, σ 2 The power of additive white Gaussian noise is expressed, and the communication rate R between the UAV and vehicle u in time slot t is obtained according to Shannon's formula. u [t] is represented as:

[0090] R u [t] = Wlog2(1+Γ) u [t]) (3-13)

[0091] Where W is the system channel bandwidth.

[0092] In an unmanned aerial vehicle (UAV)-assisted communication system, the total power E[t] of the UAV in time slot t can be modeled as the UAV's transmit power p[t] and flight power E. prop A linear combination of [t], i.e.

[0093] E[t] = p[t] + E prop [t] (3-14)

[0094] The flight power depends on the flight dynamics characteristics of the UAV, including power consumption in different flight states such as hovering, maneuvering and cruising, and can be expressed as:

[0095]

[0096] where P0 is the rotor section power, v[t] is the flight speed, v0 is the average induced speed of the rotor, P i is the induced power in the hovering state, U tip is the tip speed of the rotor blade, d0 is the fuselage drag ratio, ρ is the air density, s is the rotor solid rate, and A is the rotor disc area.

[0097] Since the optimization problem is a non-convex optimization problem containing UAV trajectory variables and launch power variables, and the vehicle matching problem is also considered as an NP-Hard problem. Therefore, reinforcement learning may lead to training difficulty to converge in such a high-dimensional action space, so a clustering-assisted reinforcement learning framework is proposed to solve the problem, which divides the problem into two sub-problems to solve;

[0098] Vehicle association matching: The vehicle flow prediction result is preprocessed by a clustering algorithm, which divides the vehicles in the system into multiple clusters, and matches the UAV with the clusters, thereby solving the vehicle association problem.

[0099] UAV trajectory design and power allocation: the UAV is trained as an agent using the MATD3 algorithm, which plans the flight trajectory and optimizes the power allocation to optimize the flight path and power settings.

[0100] The vehicle networking model in S2 satisfies the following constraint conditions: each vehicle is served by only one UAV; the UAV makes a trajectory planning decision at each decision time slot; and the UAV communication coverage area is a predefined geographical range.

[0101] The S3 specifically includes:

[0102] S31: cluster the ground vehicles using a Gaussian mixture model to determine the initial deployment position of the UAV;

[0103] S32: establish a three-dimensional coordinate model to calculate the distance between the UAV and the vehicle, the line-of-sight transmission probability and the channel gain;

[0104] S33: construct a non-convex optimization problem containing UAV trajectory variables and launch power variables;

[0105] S34: A multi-agent reinforcement learning algorithm is used to jointly optimize the trajectory planning and power allocation of the UAV.

[0106] Each cluster to which the vehicle belongs in S3 is denoted by K, where K = {1, 2,..., K}, and the U ground vehicles are divided into K sets, and the vehicles in each cluster establish a connection with the assigned UAV. To achieve efficient cluster division, this paper uses a prediction algorithm to obtain the vehicle distribution at future time, and assumes that the spatial distribution of vehicles obeys Gaussian distribution. The GMM model is used to model the position of the vehicle, and the EM algorithm is used to estimate the parameters to optimize the effect of cluster division. The EM algorithm process is shown in the table, by constantly updating the expectation step and the maximization step, the optimal modeling of the vehicle position is finally realized, and accurate cluster division results are provided.

[0107]

[0108]

[0109] The probability distribution function of vehicle u can be represented by the following formula:

[0110]

[0111] where μ k ,Σ k are Gaussian distribution variables, π k is the mixing coefficient, a new variable z is introduced, z k can only take 0 or 1, z k = 1 represents the probability of being selected by the kth class, and vice versa, z k = 0 represents the probability of not being selected by the kth class. Assuming that the vehicles are independent and identically distributed, the joint probability distribution of z k can be represented as

[0112]

[0113] Since the sample points belonging to the kth class obey Gaussian distribution, the probability density function is shown in formula (19).

[0114]

[0115] According to Bayes formula, the posterior probability p(z|u) can be obtained, and it is defined as the symbol γ(z k ) represents the posterior probability of the kth component:

[0116]

[0117] Formula (17) can be rewritten as the following expression, and the three parameters π, μ, Σ are estimated by EM algorithm:

[0118]

[0119] After the iterative optimization of the clustering algorithm, U vehicles are divided into K clusters, and M UAVs establish communication connections with these clusters respectively;

[0120] The EM algorithm is used in S31 to estimate the parameters of the Gaussian mixture model, including: initializing the number of mixture components and the parameters of each component; iteratively performing the expectation step to calculate the posterior probability, and the maximization step to update the mean, covariance and mixing coefficient; determining the vehicle cluster division and center position according to the converged model parameters.

[0121] The multi-agent extended Markov decision process framework is used in S3 to model the sub-problems, and the state space, action space, and reward function are introduced below.

[0122] State space:

[0123] State space S of UAV m m [t] is the coordinate position of UAV m at time slot t:

[0124]

[0125] Therefore, the global state of the system is represented as:

[0126] Action space:

[0127] Action space of UAV m is:

[0128]

[0129] where the action space is a continuous variable.

[0130] The joint action space of all UAVs is represented as:

[0131] Reward function

[0132] The reward function of the UAV is set so that the agent can maximize the communication rate while reducing energy consumption in the environment, and needs to satisfy the constraint conditions, so the reward function is set as:

[0133]

[0134] where Penalty is the penalty term for violating constraints C2 and C4 and C5, as shown in equation (27), and ζ is the penalty factor.

[0135]

[0136] The state space of the MATD3 algorithm is defined as the coordinate position of the UAV in the time slot, the action space includes the three-dimensional motion vector and the transmission power setting of the UAV, and the design of the reward function includes positive reward based on the average communication rate of the system; negative punishment based on the total energy consumption of the UAV; and an additional penalty term for constraint violation.

[0137] The multi-agent reinforcement learning algorithm in S34 adopts the MATD3 algorithm, including: configuring an independent Actor network and a double Critic network for each UAV; the Critic network accesses the state and action information of all UAVs during training; a delayed policy update mechanism and an action noise clipping technique are adopted; the reward function design considers the maximization of communication rate, minimization of energy consumption, and satisfaction of constraint conditions.

[0138] Specifically: each cluster to which the vehicle belongs in S3 is represented by K, where K={1,2,...,K}, and the MATD3 algorithm is implemented in a multi-agent manner based on double-delayed deep deterministic policy gradient (DDPG), which is an improvement of the TD3 algorithm. All agents have independent Actor networks and double Critic networks, and the Critic network can access the state and action of all agents during training to maximize the cumulative return of each agent. During execution, each agent makes independent decisions and actions.

[0139] Each agent u has an Actor network represented by parameters θ u , denoted as μ u , and two Critic networks represented by parameters φ u,1 and φ u,2 , denoted as Q u,1 and Q u,2 . The target networks are denoted as μ'1, Q' u,1 and Q' u,1 , respectively. Each Critic network can obtain the policy information of other agents.

[0140] The Critic network Q u,k of each agent (k=1,2) is updated by the loss function as follows:

[0141] L(φ u,k )=E[(y-Q u,k (s,a1,...,a U )) 2 ] (28)

[0142] where y is represented as:

[0143]

[0144] In equation (29), γ is the discount factor. It is obtained using cropping learning and represented as:

[0145]

[0146] Where a low and a high This indicates the upper and lower bounds of the selected action, with h and -h representing noise. upper and lower boundaries,

[0147] Each agent's Actor network μ u The updated policy gradient is:

[0148]

[0149] The overall complexity of the algorithm consists of the GMM clustering algorithm and the MATD3 algorithm. The GMM clustering algorithm is used to solve the matching problem between UAVs and vehicles. First, assuming the maximum number of iterations for the GMM clustering algorithm is I, the maximum computational complexity can be written as O(IMK). The complexity of the MATD3 algorithm mainly consists of the Actor network and the Critic network, using L... a and L c These represent the number of neuron layers in the Actor network and the Critic network, respectively, and are used accordingly. and These represent the l-th generation of the Actor network. a The number of neurons in the layers and the Critic network and the lth c Given the number of neurons in a layer, assuming that T time slots and E rounds are trained before convergence, the algorithm complexity can be expressed as:

[0150]

[0151] The overall computational complexity of the system can be expressed as:

[0152]

[0153] The specific process of the MATD3 algorithm is shown in the table below.

[0154]

[0155] In one specific embodiment, the effectiveness of the proposed scheme was verified through simulation experiments, and its performance was compared and analyzed with that of the benchmark algorithm.

[0156] The experiment was carried out in the Python 3.8 and PyTorch 1.10 environment, and the hardware configuration of the experimental computer was Intel i5-12400 CPU and Nvidia RTX 4060 GPU and 32 GB running memory.

[0157] The simulation scenario is shown in Table 2 Simulation Parameter Settings, which contains several vehicles and 4 unmanned aerial vehicles. The maximum transmission power of the unmanned aerial vehicle communication is 30 dBm, and the coverage area is a square with a side length of 1800m. In the training process, each iteration contains 100 time slots. Table 2 shows the settings of other related parameters for this simulation:

[0158]

[0159] This dataset contains vehicle quantity data from 2015 to 2017, and records the hourly vehicle flow in different areas at different time points collected by sensors in the area. This dataset mainly includes the following four aspects of information:

[0160] (1) Area ID: contains the ID information of the area.

[0161] (2) Record time: the time node at which the sensor collects each record.

[0162] (3) Vehicle flow: the number of vehicles appearing in the area at the record time node.

[0163] (4) Identification ID: the unique identification ID of each data.

[0164] To evaluate the performance of the proposed prediction model, a comparative experiment was designed, and LSTM, MLP, GRU and CNN were selected to build models for systematic comparison. The prediction performance indicators of each model under the same test set are shown in Table 4:

[0165]

[0166]

[0167] Figure 7The key evaluation indicators obtained by using different clustering algorithms under different vehicle quantity conditions are shown, including the standard deviation of the number of vehicles in each cluster and the root mean square horizontal distance of vehicles to the cluster center. In order to comprehensively evaluate the clustering effect, the GMM clustering algorithm is compared and analyzed with the hierarchical clustering and K-Means clustering algorithm. The experimental results show that, since the GMM clustering algorithm fully considers the prior probability and variance information of the data in the modeling process, compared with the K-Means clustering algorithm which only focuses on the internal distance, the clustering effect is better. In addition, compared with hierarchical clustering, GMM also shows better clustering balance and lower clustering error, thus showing obvious advantages in SDNV and RMSHD indicators. The GMM clustering algorithm can effectively identify the potential distribution pattern in the data by modeling the probability distribution of the data, thereby realizing more reasonable cluster division. In this experiment, the GMM clustering algorithm divides the data set into four different clusters, and the center coordinates of each cluster are calculated as [584.04, -311.52], [-452.23, 356.19], [-284.72, -475.60] and [621.91, 584.77], and the number of vehicles contained in each cluster is 20, 21, 20 and 17 respectively.

[0168] The flight trajectories of the multiple UAVs during execution are shown in FIG. 8. In order to more clearly present the flight path of the UAV, the Y-axis range in the figure is set to -900m to 700m. The initial position is the pre-deployment position of the UAV according to the cluster center, and the termination position corresponds to the position of the UAV at the end of the round. Among them, the UAVs move from the coordinates [584.04, -311.52] to the coordinates [450.81, -283.92], from the coordinates [-452.23, 356.19] to the coordinates [-535.82, 348.17], from the coordinates [-284.72, -475.60] to the coordinates [-301.21, -503.56], and from the coordinates [621.91, 584.77] to the coordinates [557.92, 494.79]. From the trajectory results, it can be observed that, since the movement of the UAV is limited by the energy consumption constraint, the optimized strategy after convergence tends to fly along a straight line path to balance the energy efficiency and task target in one round.

[0169] Figure 9The reward function convergence curves of multi-agent reinforcement learning algorithms in the training process are shown. The experimental results show that each algorithm enters a stable convergence stage after about 300 rounds of training, among which the MATD3 algorithm shows superior convergence characteristics and achieves the highest average reward, with an average reward value about 2% higher than that of the MADDPG algorithm. This advantage is mainly due to the MATD3 algorithm, which introduces a double Critic network and a delayed policy update mechanism, effectively suppressing the policy bias caused by Q-value overestimation in the MADDPG algorithm, thereby accelerating convergence and improving decision stability.

[0170] In addition, in order to further evaluate the influence of pre-deployment strategy on training effect, Figure 10 The convergence performance of the MATD3 algorithm without pre-deployment is compared. The experimental results show that without the pre-deployment strategy, the average reward value of the MATD3 algorithm decreases by about 33% compared to the method with the pre-deployment strategy. This result shows that the optimization of the initial position of the unmanned aerial vehicle by the clustering center of the clustering algorithm can not only effectively reduce the dimension of the search space and reduce the computational complexity of algorithm training, but also improve the training stability and accelerate the convergence process, thereby further optimizing the overall performance of multi-agent cooperative decision-making.

[0171] The average reward value changes of the MATD3 algorithm under different learning rates are shown in Figure 11 In the experimental configuration, both the Actor network and the Critic network use a three-layer fully connected neural network architecture, each hidden layer contains 256 neurons, the activation function uses ReLU, and the parameter update uses the Adam optimizer to improve training stability and convergence efficiency. In order to balance exploration and utilization, the action exploration noise coefficient is set to 0.3 at the beginning of the experiment, and an exponential decay strategy is introduced to gradually reduce the noise level, with a decay rate of 0.995 and a minimum noise limit of 0.05 to ensure that the model can focus more on policy optimization rather than random exploration in the later training period. The experimental results show that when the learning rate is set to 0.01, the average reward value of the MATD3 algorithm reaches the optimal value, which is significantly better than other learning rate settings, indicating that this learning rate configuration can effectively improve training efficiency and promote policy performance while ensuring convergence stability.

[0172] Under different transmit power constraints, the average communication rate of the vehicle changes as shown in Figure 12The experimental data show that the MATD3 algorithm based on multi-agent reinforcement learning has a performance improvement of about 3.5% in the average communication rate index compared with the MADDPG algorithm. The optimization efficiency of the genetic algorithm and the particle swarm algorithm is lower than that of the reinforcement learning method due to the limitation of global search capability. The average communication rate of the traditional greedy algorithm is significantly lower than that of the reinforcement learning method due to the lack of effective perception and optimization of the global environment in the dynamic vehicle networking scene. At the same time, the performance of the MATD3 algorithm without pre-deployment is further analyzed, which is still significantly lower than that of the other algorithms. This result further verifies the importance of the pre-deployment strategy, which can not only effectively reduce the search space, but also improve the overall communication performance of the system.

[0173] Figure 13 The comparison results of the energy efficiency of the system of various algorithms are shown. The vertical axis in the figure represents the ratio of the average communication rate of the system to the total energy consumption, which is an important indicator of overall energy utilization efficiency. The experimental results show that as the vehicle density increases, the UAV needs to allocate limited transmission power among more vehicles. Due to the limitation of the maximum transmission power constraint of the system, the average power obtained by a single vehicle decreases, which in turn leads to a decrease in the overall communication rate. In addition, due to the use of a fixed pre-deployment strategy, the increase in the number of vehicles has a limited impact on the flight path of the UAV and its energy consumption, resulting in little change in the total energy consumption of the system, and the overall energy efficiency shows a downward trend. From the performance of different algorithms, the MATD3 algorithm combined with the pre-deployment strategy performs best among all comparison schemes, and the system energy efficiency is always better than that of other algorithms. In contrast, the genetic algorithm and the particle swarm algorithm based on heuristic search are limited by global search capability and convergence efficiency, and the optimization effect is poor. The greedy algorithm cannot fully utilize global environmental information for optimization due to its local optimal search, so its performance is worse than that of other algorithms. In addition, it can be found that the performance of the MATD3 algorithm without pre-deployment strategy is significantly lower than that of the other algorithms. This is mainly because the initial position of the UAV is closer to the optimal solution through a reasonable pre-deployment strategy, and the reinforcement learning model without initial guidance is prone to suboptimal solutions in high-dimensional continuous space, resulting in increased system energy consumption and decreased average communication rate. Therefore, this shows that the reinforcement learning method combined with the pre-deployment strategy can more effectively improve the energy efficiency of the system and has better adaptability and robustness in high-density vehicle environments.

[0174] The effectiveness of the proposed scheme is verified through simulation experiments, and performance comparison analysis is performed with benchmark algorithms. The experimental results show that the method has significant improvement in system average rate and energy efficiency.

[0175] An unmanned aerial vehicle assisted vehicle networking system, comprising: a distributed sensor network for collecting multi-source historical traffic data;

[0176] a cloud platform for implementing step S1;

[0177] a plurality of unmanned aerial vehicles configured with communication base stations for executing the generated optimized trajectory;

[0178] a vehicle terminal device for establishing a communication connection with the unmanned aerial vehicle.

[0179] The above description is only the preferred embodiment of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can make equivalent replacements or changes within the technical range disclosed by the present application according to the technical solution and the inventive concept of the present application, which should be covered within the protection scope of the present application.

Claims

1. A drone-assisted vehicle-to-everything (V2X) trajectory planning method based on traffic flow prediction, characterized in that, Includes the following steps: S1: Using multi-source historical traffic data collected by a distributed sensor network as input, a traffic flow spatiotemporal prediction model is established based on a deep learning algorithm; the multi-source historical traffic data is uploaded to a cloud platform or a centralized data processing node in real time or periodically through a wireless communication network, and after data cleaning, integration and analysis, prediction results are generated; S2: A vehicle-to-everything (V2X) model consisting of multiple vehicles and multiple drones is constructed. M drones are deployed, and U vehicles are randomly distributed within the area. Let M be the set of drones. Each drone makes autonomous decisions within each time slot t to optimize its flight path and select the best service target; S3: Establish a drone trajectory optimization method based on traffic flow prediction data, specifically including: combining traffic flow prediction data and using clustering algorithms to cluster ground vehicles, using the cluster center as the pre-deployment location of drones for each vehicle cluster, and further jointly optimizing drone trajectory planning and resource allocation strategies. The MATD3 algorithm is used to optimize the three-dimensional trajectory planning of the UAV and determine the UAV's flight path and transmission power.

2. The UAV-assisted vehicle-to-everything (V2X) trajectory planning method based on traffic flow prediction according to claim 1, characterized in that: The entire data processing described in S1 includes data collection, preprocessing, model training, and predictive model evaluation; In the preprocessing stage, historical data are systematically evaluated, the distribution characteristics of missing data and their potential impact are analyzed, and the missing data are repaired using the mean imputation method. After imputing the missing values, various data transformation strategies are applied, and Z-score normalization is used for standardization.

3. The UAV-assisted vehicle-to-everything (V2X) trajectory planning method based on traffic flow prediction according to claim 1, characterized in that: In S1, after data preprocessing, prediction models are constructed by selecting LSTM, GRU, CNN and multilayer perceptron networks based on the temporal characteristics and complex relationships of the data. The dataset is divided into training set and test set for model training and optimization, and the generalization ability of the model is evaluated on the test set.

4. The UAV-assisted vehicle-to-everything (V2X) trajectory planning method based on traffic flow prediction according to claim 1, characterized in that: The vehicle-to-everything (V2X) model described in S2 satisfies the following constraint: each vehicle is served by only one drone; The drone makes trajectory planning decisions within each decision time slot; the drone's communication coverage area is a predefined geographical range.

5. The UAV-assisted vehicle-to-everything (V2X) trajectory planning method based on traffic flow prediction according to claim 3, characterized in that: S3 specifically includes: S31: Use a Gaussian mixture model to cluster ground vehicles to determine the initial deployment location of the UAV; S32: Establish a three-dimensional coordinate model and calculate the distance between the UAV and the vehicle, the line-of-sight transmission probability, and the channel gain; S33: Construct a nonconvex optimization problem that includes UAV trajectory variables and transmit power variables; S34: A multi-agent reinforcement learning algorithm is used to jointly optimize the trajectory planning and power allocation of the UAV.

6. The UAV-assisted vehicle-to-everything (V2X) trajectory planning method based on traffic flow prediction according to claim 5, characterized in that: The multi-agent reinforcement learning algorithm described in S34 adopts the MATD3 algorithm, which includes: configuring an independent Actor network and a dual Critic network for each UAV; the Critic network accessing the state and action information of all UAVs during training; adopting a delayed policy update mechanism and action noise pruning technique; and designing the reward function to simultaneously consider maximizing communication rate, minimizing energy consumption, and satisfying constraints.

7. The UAV-assisted vehicle-to-everything (V2X) trajectory planning method based on traffic flow prediction according to claim 6, characterized in that: The state space of the MATD3 algorithm is defined as the coordinate position of the UAV in the time slot, the action space includes the UAV's three-dimensional motion vector and transmission power settings, and the reward function is designed to include positive rewards based on the system's average communication rate; negative penalties based on the UAV's total energy consumption; and additional penalty terms for constraint violations.

8. The UAV-assisted vehicle-to-everything (V2X) trajectory planning method based on traffic flow prediction according to claim 5, characterized in that: In S31, the EM algorithm is used to estimate the parameters of the Gaussian mixture model, including: initializing the number of mixture components and the parameters of each component; iteratively executing the expectation step to calculate the posterior probability, and maximizing the step to update the mean, covariance and mixture coefficients; and determining the vehicle cluster division and center position based on the converged model parameters.

9. A drone-assisted vehicle networking system, characterized in that, include: Distributed sensor networks are used to collect multi-source historical traffic data; A cloud platform for implementing step S1 of the method according to any one of claims 1-8; Multiple drones equipped with communication base stations are used to execute the optimized trajectory generated by the method described in any one of claims 1-8; Vehicle-mounted terminal equipment is used to establish communication connections with drones.

Images