User trajectory prediction and unmanned aerial vehicle path planning method in real geographical environment
By using deep learning LSTM network to predict user trajectories in real geographical environment and planning drone paths in reinforcement learning DDQN network, the problems of inaccurate user trajectory prediction and discontinuous drone trajectory in the existing technology are solved, and the problem of inaccurate user trajectory prediction and discontinuous drone trajectory in the existing technology is achieved. High total system throughput.
Patent Information
- Application Number
- CN202410774904.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-17
- Publication Date
- 2024-11-08
AI Technical Summary
The existing drone path planning methods fail to effectively consider the real geographical environment, resulting in inaccurate user trajectory prediction and discontinuous drone trajectory designed, and failure to reach the calculated next position within one time slot.
Deep learning LSTM network is used to predict user trajectories in real geographical environment, and to use reinforcement learning DDQN network to plan the three-dimensional path of the drone, considering the correlation between drone positions and dynamic flight time slot duration to maximize Total throughput of the system.
More accurate user trajectory prediction and continuous drone three-dimensional path planning are achieved, significantly improving the total system throughput.
Smart Images

Figure CN118915814A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of wireless communication technology and flight control of unmanned aerial vehicles (UAVs), and relates to ground user trajectory prediction and three-dimensional path planning of unmanned aerial vehicles, especially predicting the user's trajectory in a real geographical environment, and planning the path of the unmanned aerial vehicle on this basis. The present invention is based on a deep learning long short-term memory (LSTM) network, and performs prediction on a user's historical trajectory data set. Based on the predicted trajectories of multiple ground users, the present invention applies a reinforcement learning double deep Q network (DDQN), considers the correlation of the positions of unmanned aerial vehicles between adjacent time slots, and plans a three-dimensional path of the unmanned aerial vehicle with the maximum total throughput as the optimization goal, realizing user trajectory prediction and three-dimensional path planning of unmanned aerial vehicles in a real geographical environment, and providing ground users with better information rate and signal coverage services. Background Art
[0002] In recent years, drones have been widely used in auxiliary communication tasks as a new type of base station. When the ground fixed base station (GBS) fails, the drone aerial base station (ABS) can provide temporary communication services for ground users (GU). At the same time, the drone aerial base station also has the characteristics of strong mobility, flexible deployment, and ignoring terrain obstacles. It effectively improves the problem that the ground base station signal is difficult to reach in complex environments, and plays an important role in traffic unloading, post-disaster rescue and other scenarios. At present, there have been many studies on drone-assisted communication. For example, Ruikang Zhong, Xiao Liu and Haibo Mei, Kun Yang optimized the trajectory of drones in their papers "Multi-Agent Reinforcement Learning in NOMA-Aided UAV Networks for Cellular Offloading" and "3D-Trajectory and Phase-Shift Design for RIS-Assisted UAV Systems Using Deep Reinforcement Learning" to provide better communication services for mobile ground users. However, these two papers modeled the user's movement as a directional walking model and a random movement model, respectively, and neither of these two movement models considered the real geographical environment. Inspired by this, it is of great significance to obtain the user trajectory that conforms to the real geographical environment and plan the path of the UAV based on it.
[0003] Thanks to the rapid development of neural networks, many neural network prediction methods have been proposed, and the problem of predicting user trajectories has been well solved. For example, Chujie Wang and Lin Ma used support vector regression (SVR), gated recurrent unit (GRU), LSTM and other networks in their paper "Exploring Trajectory Prediction Through Machine Learning Methods" to predict user trajectories based on user historical trajectory data. Ya Liu and Hongwen Yang used deep learning LSTM networks to predict user trajectories in their paper "Mobile User Trajectory Prediction Based on Machine Learning" and obtained relatively good prediction results, proving the ability of deep learning LSTM networks in predicting user trajectories.
[0004] Some works have also studied the content of joint user trajectory prediction and UAV three-dimensional path planning. Based on the ground user trajectory prediction results obtained by the echo state network (ESN), Xiao Liu and Yuanwei Liu planned the path of the UAV in "Trajectory Design and Power Control for Multi-UAV Assisted Wireless Networks: A Machine Learning Approach". However, they only provided the planned two-dimensional path of the UAV, did not provide the three-dimensional spatial movement path, and did not consider whether the UAV can reach the calculated next position within a time slot. Jinjing Wang used convolutional neural network (CNN) to extract user motion features and predict user trajectories in "Trajectory Planning of UAV-Enabled Data Uploading for Large-Scale Dynamic Networks: A Trend Prediction Based Learning Approach", and proposed a framework that combines user trajectory prediction with UAV path planning. However, the user's historical movement trajectory is simulated using the Gauss-Markov Random Model (GMRM), and the performance of network prediction is not provided. In "Rendezvous: Opportunistic Data Delivery to Mobile Users by UAVs Through TargetTrajectoryPrediction" by Jinji Yoon and A-Hyun Lee, drones plan paths based on the predicted trajectories of users. The trajectories of all users collected are divided into several clusters according to similarity, and each cluster is regarded as a representative trajectory of similar user trajectories. For each target mobile user, the initial positions are matched with several clusters to obtain the predicted trajectory, but this method obtains the user's possible future location, not the exact location, so it is not reasonable.
[0005] Based on the above research status, it can be concluded that traditional drone path planning methods often model the user movement model as a simple random movement model without considering the real geographical environment information. The research on drone path planning based on predicting user trajectories also has problems such as inaccurate user position prediction and discontinuous designed drone trajectories. Based on the user historical trajectory data in a real geographical environment, the present invention adopts a deep learning LSTM network to predict user trajectories. Based on the user predicted trajectory, a reinforcement learning DDQN network is used to plan the three-dimensional path of the drone to effectively improve the total throughput. Summary of the invention
[0006] To solve the above problems, the present invention proposes a method for user trajectory prediction and drone path planning in a real geographical environment, taking into account the correlation between the real geographical environment and the drone position between adjacent time slots. Based on multiple ground user historical trajectory datasets established in a real geographical environment, a deep learning LSTM network is used to predict the trajectories of multiple ground users. Based on the predicted trajectories of multiple ground users, a reinforcement learning DDQN network is used to plan a three-dimensional drone path and optimize the total system throughput.
[0007] Innovation of the present invention
[0008] First, the present invention proposes a method for user trajectory prediction and drone path planning in a real geographical environment in response to the unreasonable situation of ground user trajectories in existing studies, such as those that do not conform to actual roads, buildings, obstacles, etc. This method uses the user historical trajectory data established in a real geographical environment as a data set, uses a deep learning LSTM network to predict ground user trajectories, and predicts ground user trajectories that conform to the real geographical environment by analyzing the user coordinate position data sampled in chronological order.
[0009] Second, based on the user's predicted trajectory, the present invention plans the three-dimensional path of the drone based on the deep learning DDQN network. Considering the range of the drone's flight area and the correlation of the drone's position between adjacent time slots, a three-dimensional drone path that conforms to the movement trend of the user group is planned. Compared with existing research, the user trajectory considered in the present invention is reasonable and conforms to the real geographical environment, rather than a simple random movement. At the same time, unlike existing research that does not consider whether the drone can reach the calculated next position within a time slot, the present invention considers the correlation of the drone's position between adjacent time slots by setting a dynamic flight time slot duration.
[0010] Third, the effectiveness of the user trajectory prediction and UAV path planning method in a real geographical environment proposed by the present invention has been verified through experiments. Compared with existing trajectory prediction studies, the deep learning LSTM network method for predicting user trajectories used in the present invention achieves lower prediction errors and can more accurately predict user trajectories. Furthermore, based on the user's predicted trajectory, considering the correlation between the positions of UAVs in adjacent time slots, the present invention uses a deep learning DDQN network UAV three-dimensional path planning method to obtain a continuous UAV three-dimensional path that conforms to the movement trend of the user group, and compared with other UAV path planning methods, the present invention achieves a higher total system throughput. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] Figure 1 It is a system model diagram of a UAV base station provided by an embodiment of the present invention providing communication services for multiple mobile ground users;
[0012] Figure 2 is a system flow chart of joint user trajectory prediction and drone base station three-dimensional path planning provided by an embodiment of the present invention;
[0013] Figure 3 It is a moving trajectory diagram of multiple users on a real satellite map predicted by a deep learning LSTM network provided by an embodiment of the present invention;
[0014] Figure 4 This is a graph showing the variation of the error index of the user trajectory predicted based on the deep learning LSTM network with different total time slot numbers provided by an embodiment of the present invention;
[0015] Figure 5 The embodiment of the present invention provides a three-dimensional path map of a UAV planned by using a reinforcement learning DDQN network based on the predicted trajectories of multiple ground users;
[0016] Figure 6 A two-dimensional trajectory map of a UAV is provided by an embodiment of the present invention, based on the predicted trajectories of multiple ground users, using a reinforcement learning DDQN network for planning;
[0017] Figure 7 The embodiment of the present invention provides a variation of the total system throughput with different total time slot numbers and different UAV transmission powers when different UAV path planning methods are used based on the predicted trajectories of multiple ground users; DETAILED DESCRIPTION
[0018] like Figure 1 The UAV communication system shown in FIG. 1 and the method for user trajectory prediction and UAV path planning in a real geographical environment proposed by the present invention refer to FIG. Figure 2 , specifically including the following steps:
[0019] Step 1: Establish a system model consisting of drones and ground users in the drone communication network. The drone acts as an aerial base station and provides communication services to K ground mobile users during flight. The entire flight time of the drone is divided into N time slots, and the duration of the nth time slot is τ(n). In time slot n, the three-dimensional position coordinates of the drone are represented as s U (n) = [x U (n),y U (n),z U (n)], where x U (n), y U (n), z U (n) represent the x-axis, y-axis, and z-axis coordinates of the drone, respectively. The speed of the drone is V. The drone will adjust its flight direction according to the user's movement, thereby providing better information rate and signal coverage services. Frequency division multiple access is used for data transmission between the drone and the user. K users move at a speed of v according to the actual geographical environment and road conditions. The entire movement time of the user is evenly divided into M time periods, each of which lasts for u, and the user's position at the beginning of each time period is taken as the user's position in that time period. The three-dimensional coordinates of the kth user in the mth time period are δ k (m) = [x k (m),y k (m),z k (m)], where x k (m), y k (m), z k (m) represents the x-axis, y-axis, and z-axis coordinates of the user respectively;
[0020] Step 2 is as follows:
[0021] (2a) Calculate the probability that the communication link between the UAV and the user is a line-of-sight (LoS) link or a non-line-of-sight (NLoS) link. Specifically, the probability that the communication link between the UAV and user k in time slot n is a LoS link is for:
[0022]
[0023] Among them, a and b are constants related to the channel, is the horizontal distance between the UAV and user k, D k (n)=||s U (n)-δ k (m)|| represents the three-dimensional distance between the drone and user k. ||·|| represents the binary norm of the vector. Indicates the index of the user mobility time period corresponding to the nth time slot, The probability that the communication link between the UAV and user k in time slot n is an NLoS link is:
[0024] (2b) Calculate the average path loss of the communication link between the drone and the user. Specifically, the path loss when the communication link between the drone and user k in time slot n is a LoS link is Calculated as:
[0025]
[0026] Path loss when the communication link between the drone and user k in time slot n is an NLoS link Calculated as:
[0027]
[0028] Where f is the carrier frequency, c is the speed of light, η LoS and η NLoS are the average additional loss constants for LoS and NLoS links, respectively. Therefore, the average path loss L of the communication link between the drone and user k in time slot n is k (n) can be expressed as:
[0029]
[0030] (2c) Calculate the information rate of the communication link between the drone and the user. The information rate between the drone and user k in time slot n is: Where W is the total system bandwidth, P k is the transmission power from the UAV to user k, σ 2 is the power of Gaussian white noise with zero mean, L k (n) is the average path loss between the UAV and user k in time slot n;
[0031] Step 3, UAV-assisted communication combined with user trajectory prediction and UAV 3D path planning in a real geographic environment, establishes the total throughput maximization problem as follows:
[0032]
[0033] in, represents the total throughput, S = {s U (n),n∈N} is the user coordinate set, T = {τ(n),n∈N} is the UAV flight time slot set. (a), (b), (c) are the three-dimensional coordinate constraints of the UAV, which constrain the flight area of the UAV, x min 、x max are the minimum and maximum values of the horizontal coordinate of the drone, ymin ,y max are the minimum and maximum values of the vertical coordinate of the drone, z min 、z max are the minimum and maximum values of the vertical coordinates of the drone, respectively. In (d), τ(n) is min and τ max The duration of the nth time period selected between;
[0034] Step 4: Based on the user historical trajectory dataset, use the deep learning LSTM network to train and predict user trajectories. The specific process is as follows:
[0035] (4a) The user's historical movement trajectory can be obtained by social media software that provides positioning services. The obtained data set is divided into a training set and a test set in a ratio of 75% and 25%, and input into the LSTM network;
[0036] (4b) After the LSTM network receives the user's historical trajectory data, it uses the forget gate composed of the sigmoid activation function and the dot multiplication operation to decide which information should be remembered and which information should be forgotten. It uses the input gate composed of the sigmoid activation function, the tanh hyperbolic sine function, the dot multiplication operation, and the addition operation to decide which new information to retain. It uses the output gate composed of the sigmoid activation function, the tanh hyperbolic sine function, and the dot multiplication operation to decide which information to output;
[0037] (4c) Select Mean Square Error (MSE) as the loss function to minimize the true user location δ k (i) and predicting user location The mean square error between k (i) represents the i-th data in the training data set, and the loss function is defined as:
[0038]
[0039] Where I is the total number of data in the training data set;
[0040] (4d) The following three error indicators are calculated to evaluate the performance of the deep learning LSTM network in predicting user trajectories, including MSE, root mean square error (RMSE) and mean absolute error (MAE).
[0041] Step 5: Based on the user’s predicted trajectory, the reinforcement learning DDQN network is used to establish a reward function that meets the goal and calculate the three-dimensional path of the drone, as follows:
[0042] (5a) In the target scene, import the movement trajectory of the ground user predicted in step 4;
[0043] (5b) Building an intelligent agent: Building the drone as an independent intelligent agent;
[0044] (5c) Define the state: The coordinates of the drone in each time slot are defined as the state s(n) of the agent, i.e., s(n) = s U (n);
[0045] (5d) Action definition: The action that the agent can choose in each time slot is represented by a(n) = (κ(n), τ(n)), which is divided into two parts: flight direction selection κ(n) and flight time slot duration τ(n). κ(n) contains seven actions, defined as:
[0046] κ(n)∈{east, west, north, south, up, down, hover}
[0047] (5e) Initialize the DDQN network experience pool F, the maximum number of iterations E, the state of the agent s(n), and initialize the parameters w of the evaluation Q network and the target Q network e and w t , and set w e =w t ;
[0048] (5f) Calculate the action corresponding to the maximum Q value: DDQN contains two Q networks, namely the evaluation Q network and the target Q network. To reduce the error caused by overestimation of Q values, DDQN finds the action a corresponding to the maximum Q value from the evaluation Q network. * (n+1), select the Q value in the target Q network according to this action, this action a * (n+1) is calculated as:
[0049] a * (n+1)=argmax a(n) Q e (s(n+1);w e ),
[0050] Among them, Q e represents the evaluation Q network, s(n+1) represents the state of the drone in the n+1 time slot, Q e (s(n+1);w e ) represents the Q value corresponding to different actions calculated by the Q network under the state s(n+1), argmax a(n) Q e (s(n+1);w e ) means finding the action corresponding to the maximum Q value among the Q values corresponding to multiple different actions;
[0051] (5g) Establish an action selection strategy: Use the greedy strategy to select actions, that is, the drone randomly selects an action with probability ò and selects a with probability 1-ò * (n+1), where ò decays gradually from large to small, so that the action gradually shifts from exploring a larger space to obtaining a larger reward;
[0052] (5h) Construct reward function: After the agent performs an action, it will receive the reward corresponding to the action as the basis for action optimization. When the agent completes the action and its position does not exceed the set flight area, the reward function is:
[0053]
[0054] in, Represents the total throughput of all users in time slot n. If the flight area is exceeded, the reward function is 1% r(n);
[0055] (5i) Calculate the target Q value: The target Q value ξ(n) is calculated by the following formula:
[0056] ξ(n)=r(n)+γQ t (s(n+1),a*(n+1);w t ),
[0057] Among them, Q t represents the target Q network, γ represents the discount factor, Q t (s(n+1),a*(n+1);w t ) indicates action a in time slot n+1 * (n+1) The corresponding Q value in the target Q network;
[0058] (5j) Calculate the Loss function value: Loss function L(w e ) is calculated as:
[0059] L(w e )=E[ξ(n)-Q e (s(n),a(n);w e )],
[0060] Among them, Q e (s(n),a(n);w e ) is the Q value obtained by evaluating the Q network in time slot n, E[·] represents the mathematical expectation operation, and the calculation result of the Loss function is returned to the evaluation Q network Q e Middle, Q e In this way, we can learn and optimize the network parameters w e ;
[0061] (5k) Each agent selects an action as defined in (5d) and moves according to the action selection strategy described in (5g);
[0062] (5l) Calculate the reward function as described in (5h) and update the Q value Q of the evaluation Q network e (s(n),a(n);w e ) and the target Q value ξ(n), and updates w at every set fixed period. t =w e ;
[0063] (5m) Iterate steps (5f) to (5l) until the maximum number of iterations E is reached.
[0064] The effects of the present invention can be further described in combination with the following simulation conditions, contents and results:
[0065] A. Simulation Conditions
[0066] A1) The historical trajectory dataset of 6 users is divided into a training set and a test set at a ratio of 75% and 25%;
[0067] A2) A rectangular area of 800m*900m in a real geographical environment;
[0068] A3) Use Python to generate the experimental simulation scene and set the drone flight altitude range to [25m, 100m].
[0069] B. Simulation Content
[0070] B1) For six ground users, their trajectories are predicted by deep learning LSTM network and displayed on the real geographical environment satellite map. The results are as follows Figure 3 As shown;
[0071] B2) The effects of different total time slot numbers on prediction performance are compared, and the prediction performance of deep learning LSTM network with different total time slot numbers is evaluated using three error indicators: MSE, MAE, and RMSE. Figure 4 As shown;
[0072] B3) Based on the predicted trajectories of the six ground users, the reinforcement learning DDQN network is used to plan the path of the UAV and obtain its three-dimensional path, such as Figure 5 As shown;
[0073] B4) Based on the predicted trajectories of the six ground users, the reinforcement learning DDQN network is used to plan the path of the UAV and obtain a two-dimensional path corresponding to its three-dimensional path, such as Figure 6 As shown;
[0074] B5) Based on the predicted trajectories of six ground users, different UAV path planning algorithms were used: DDQN, Deep Q Network (DQN), Q-learning, random action selection, and fixed UAV. The changes in the total throughput of these five path planning algorithms when the total number of time slots and the UAV transmission power changed were compared. Figure 7 shown.
[0075] C. Simulation Results
[0076] C1) Figure 3 The result graph is a result of comparing the user's actual trajectory with the user's predicted trajectory obtained by using the deep learning LSTM network on the established user historical trajectory dataset, and displayed on a satellite map of the real geographical environment. From the simulation results, it can be seen that the six dotted lines representing the predicted trajectories of the six users are highly overlapped with the solid lines representing the real trajectories of the six users, and the trajectories are all along the actual roads, which shows that the trajectory results predicted by the deep learning LSTM network are accurate and consistent with the real geographical environment.
[0077] C2) Figure 4 This paper studies the impact of different total time slot numbers on the prediction performance of the deep learning LSTM network, and uses three error indicators, MSE, MAE, and RMSE, for evaluation. As can be seen from the result graph, as the total number of time slots increases, the three error indicators are all decreasing. This is because the total number of time slots corresponds to the user trajectory sampling time interval. The increase in the total number of time slots corresponds to a decrease in the trajectory sampling time interval, which will help to record the details and changes of the user's mobile trajectory more finely, and help the deep learning LSTM network capture the association between the recorded user locations, thereby reducing the prediction error.
[0078] C3) Figure 5 The three-dimensional path of the drone is obtained by using the reinforcement learning DDQN network to plan the path of the drone based on the predicted trajectories of six ground users. From the three-dimensional path diagram, it can be seen that the drone adjusts its direction and altitude.
[0079] C4) Figure 6 Based on the predicted trajectories of six ground users, the reinforcement learning DDQN network is used to plan the path of the drone, and the two-dimensional path corresponding to the three-dimensional path of the drone is obtained. From the two-dimensional path diagram, it can be seen that the overall trend of the drone is to move eastward, which is in line with the overall movement trend of multiple ground users, thereby improving the total throughput.
[0080] C5) Figure 7Based on the predicted trajectories of six ground users, five different drone path planning algorithms were used: DDQN, DQN, Q-learning, random selection of actions, and fixed drones. The changes in the total throughput of these five path planning algorithms when the total number of time slots and the drone transmission power changed were compared. As can be seen from the figure, with the increase of the total number of time slots and the drone transmission power, the total throughput obtained by the five path planning algorithms all showed an upward trend. Among them, LSTM-DDQN has the highest total throughput, because DDQN solves the problem of overestimation of Q values in DQN and Q-learning and introduces a more stable training state. The random selection of actions scheme fails to associate with user movement, while the fixed drone position restricts the movement of the drone. The latter two schemes do not associate the drone position with the user position, which makes the total throughput of the latter two schemes low.
[0081] In summary, the present invention proposes a method for user trajectory prediction and drone path planning in a real geographical environment. On the user historical trajectory dataset established in the real geographical environment, a deep learning LSTM network is used to train and predict the user's trajectory. The obtained predicted trajectory is applied to the drone communication scenario, and the correlation of the drone position between adjacent time slots is considered, and the dynamic flight time slot duration is set for the drone to choose. With maximizing the total system throughput as the optimization goal, a suitable reward function is set, and the reinforcement learning DDQN network is used to plan the three-dimensional path of the drone. Compared with the algorithm proposed in the present invention and the algorithms in other works, the algorithm of the present invention can achieve a significant improvement in the total throughput.
[0082] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.
Claims
1. A method for user trajectory prediction and drone path planning in a real geographical environment, characterized in that: The specific steps include: S1: Establish a system model consisting of drones and ground users in the drone communication network; S1a: The drone acts as an aerial base station and provides communication services to K ground mobile users during the flight. The entire flight time of the drone is divided into N time slots, and the duration of the nth time slot is τ(n); S1b: In time slot n, the three-dimensional position coordinates of the drone are represented as s U (n) = [x U (n),y U (n),z U (n)], where x U (n), y U (n), z U (n) represents the x-axis, y-axis, and z-axis coordinates of the drone, respectively; S1c: The speed of the drone is V. The drone will adjust its flight direction according to the user's movement to provide better information rate and signal coverage services; S1d: Frequency division multiple access is used for data transmission between drones and users. K users move at a speed v according to the real geographical environment and road conditions. The entire moving time of the user is evenly divided into M time periods. The duration of each time period is u, and the user's position at the beginning of each time period is taken as the user's position in that time period. S1e: The three-dimensional coordinates of the kth user in the mth time period are δ k (m) = [x k (m),y k (m),z k (m)], where x k (m), y k (m), z k (m) represents the x-axis, y-axis, and z-axis coordinates of the user respectively; S2: Establish the information rate between drones and users in the drone communication network. In time slot n, the information rate R between drones and user k k (n) is: Where W is the total system bandwidth, P k is the transmission power from the UAV to user k, σ 2 is the power of Gaussian white noise with zero mean, L k (n) is the average path loss between the UAV and user k in time slot n; S3: The total throughput maximization problem is established as: s.t.x min ≤x U (n)≤x max ,n=1,2,...,N, (a) y min ≤y U (n)≤y max ,n=1,2,...,N, (b) z min ≤z U (n)≤h max ,n=1,2,...,N, (c) t min ≤τ(n)≤τ max ,n=1,2,...,N. (d) in, represents the total throughput, S = {s U (n),n=1,2,...,N} is the set of UAV 3D coordinates, T={τ(n),n=1,2,...,N} is the set of UAV flight time slot durations; (a), (b), (c) are the UAV 3D position coordinate restrictions, which constrain the UAV flight area, x min 、x max are the minimum and maximum values of the x-axis coordinates of the drone, and the y min ,y max are the minimum and maximum values of the y-axis coordinates of the drone, and z min 、z max are the minimum and maximum values of the z-axis coordinates of the drone, respectively; in (d), τ(n) is the value of min and τ max The duration of the nth UAV flight time slot selected between; S4: Use the user historical trajectory dataset to train the LSTM deep learning network to obtain the user prediction trajectory; S5: Using the user predicted trajectory obtained in S4, the reinforcement learning DDQN network is used to establish a reward function that meets the optimization goal and calculate the three-dimensional path of the drone.
2. According to the method of user trajectory prediction and drone path planning in a real geographical environment in claim 1, it is characterized in that: The joint user trajectory prediction and drone aerial base station three-dimensional path planning algorithm for drone communication is based on the historical trajectory dataset of ground users in a real geographical environment. The deep learning Long Short Term Memory (LSTM) network is used to predict the discrete positions of multiple ground user trajectories. Based on the predicted user mobile positions, the reinforcement learning Double Deep Q-Network (DDQN) is used to plan the drone's path. At the same time, the correlation of the drone positions between adjacent time slots is considered, and the dynamic flight time slot duration is set to plan the drone's three-dimensional path with the maximum total throughput as the optimization goal.
3. The method for user trajectory prediction and drone path planning in a real geographical environment according to claim 1, characterized in that: In S2, establishing the information rate between the drone and the user in the drone communication network specifically includes the following steps: S2a: Calculate the probability that the communication link between the drone and the user is a line-of-sight (LoS) link or a non-line-of-sight (NLoS) link; specifically, the probability that the communication link between the drone and user k in time slot n is a LoS link for: Among them, a and b are constants related to the channel, is the horizontal distance between the UAV and user k, D k (n)=||s U (n)-δ k (m)|| represents the three-dimensional distance between the drone and user k, ||·|| represents the second norm of the vector, Indicates the index of the user mobility time period corresponding to the nth time slot, represents rounding down, the probability that the communication link between the UAV and user k in time slot n is an NLoS link is: S2b: Calculate the average path loss of the communication link between the drone and the user; specifically, the path loss when the communication link between the drone and user k in time slot n is a LoS link Calculated as: Path loss when the communication link between the drone and user k in time slot n is an NLoS link Calculated as: Where f is the carrier frequency, c is the speed of light, η LoS and η NLoS are the average additional loss constants for LoS and NLoS links respectively; therefore, the average path loss L of the communication link between the drone and user k in time slot n is k (n) can be expressed as: S2c: Calculate the information rate of the communication link between the drone and the user. The information rate between the drone and user k in time slot n is:
4. The method for user trajectory prediction and drone path planning in a real geographical environment according to claim 1, characterized in that: In the S4, the following steps are specifically performed: S4a: The user's historical movement trajectory can be obtained by social media software that provides positioning services. The obtained data set is divided into a training set and a test set in a ratio of 75% and 25%, and input into the LSTM network; S4b: After the LSTM network receives the user's historical trajectory data, it uses the forget gate composed of the sigmoid activation function and the dot multiplication operation to decide which information should be remembered and which information should be forgotten. It uses the input gate composed of the sigmoid activation function, the tanh hyperbolic sine function, the dot multiplication operation, and the addition operation to decide which new information to retain. It uses the output gate composed of the sigmoid activation function, the tanh hyperbolic sine function, and the dot multiplication operation to decide which information to output. S4c: Select Mean Square Error (MSE) as the loss function to minimize the true user location δ k (i) and predicting user location The mean square error between k (i) represents the i-th data in the training data set, and the loss function is defined as: Where I is the total number of data in the training data set; S4d: The following three error indicators are calculated to evaluate the performance of the deep learning LSTM network in predicting user trajectories, including MSE, root mean square error (RMSE) and mean absolute error (MAE).
5. The method for user trajectory prediction and drone path planning in a real geographical environment according to claim 1, characterized in that: In S5, the following steps are specifically performed: S5a: In the target scene, import the movement trajectory of the ground user predicted in step 4; S5b: Building an intelligent agent: Building the drone as an independent intelligent agent; S5c: Define the state: define the coordinates of each time slot of the drone as the state s(n) of the agent, that is, s(n) = s U (n); S5d: Define actions: The actions that the agent can choose in each time slot are represented by a(n) = (κ(n), τ(n)), which is divided into two parts: flight direction selection κ(n) and flight time slot duration τ(n). κ(n) contains seven actions, defined as: κ(n)∈{east, west, north, south, up, down, hover} S5e: Initialize the DDQN network experience pool F, the maximum number of iterations E, the state of the agent s(n), and initialize the parameters w of the evaluation Q network and the target Q network e and w t , and set w e =w t ; S5f: Calculate the action corresponding to the maximum Q value: DDQN contains two Q networks, namely the evaluation Q network and the target Q network. In order to reduce the error caused by overestimation of Q value, DDQN finds the action a corresponding to the maximum Q value from the evaluation Q network. * (n+1), select the Q value in the target Q network according to this action, this action a * (n+1) is calculated as: a * (n+1)=argmax a(n) Q e (s(n+1);w e ), Among them, Q e represents the evaluation Q network, s(n+1) represents the state of the drone in the n+1 time slot, Q e (s(n+1);w e ) represents the Q value corresponding to different actions calculated by the Q network under the state s(n+1), argmax a(n) Q e (s(n+1);w e ) means finding the action corresponding to the maximum Q value among the Q values corresponding to multiple different actions; S5g: Establish an action selection strategy: Use the greedy strategy to select actions, that is, the drone randomly selects actions with probability ò and selects a with probability 1-ò * (n+1), where ò decays gradually from large to small, so that the action gradually shifts from exploring a larger space to obtaining a larger reward; S5h: Construct reward function: After the agent performs an action, it will receive the reward corresponding to the action as the basis for action optimization. When the agent completes the action and its position does not exceed the set flight area, the reward function is: in, Represents the total throughput of all users in time slot n. If it exceeds the set flight area, the reward function is 1% r(n); S5i: Calculate the target Q value: The target Q value ξ(n) is calculated by the following formula: ξ(n)=r(n)+γQ t (s(n+1),a*(n+1);w t ), Among them, Q t represents the target Q network, γ represents the discount factor, Q t (s(n+1),a*(n+1);w t ) indicates action a in time slot n+1 * (n+1) The corresponding Q value in the target Q network; S5j: Calculate the Loss function value: Loss function L(w e ) is calculated as: L(w e )=E[ξ(n)-Q e (s(n),a(n);w e )], Among them, Q e (s(n),a(n);w e ) is the Q value obtained by evaluating the Q network in time slot n, E[·] represents the mathematical expectation operation, and the calculation result of the Loss function is returned to the evaluation Q network Q e In, Q e In this way, we can learn and optimize the network parameters w e ; S5k: Each agent selects an action as defined in S5d according to the action selection strategy as described in S5g and moves; S5l: Calculate the reward function as described in S5h, and update the Q value Q of the evaluation Q network e (s(n),a(n);w e ) and the target Q value ξ(n), and updates w at every set fixed period. t =w e ; S5m: Iterate steps S5f to S51 until the maximum number of iterations E is reached.
Citation Information
Cited By
Electricity market user load prediction method, system, equipment and medium
CN119940665A
Power satellite communication-oriented dynamic multi-network switching method and related device
CN121568183A
Unmanned aerial vehicle path planning method and system based on 5G signal coverage range
CN122219482A
