A method and system for unmanned aerial vehicle path planning based on 5G signal coverage range

Abstract

The application provides a method and system for unmanned aerial vehicle path planning based on 5G signal coverage range, which solves the technical problem that the communication rate is limited or invalid due to entering a 5G signal poor or signal-free area during the flight of the unmanned aerial vehicle. It includes periodically acquiring data samples and training a deep learning algorithm, continuously predicting the predicted value of the physical layer signal quality index in the environment based on the trained deep learning algorithm; constructing a radio map according to the predicted value of the physical layer signal quality index; and planning the flight path of the unmanned aerial vehicle based on the radio map using a reinforcement learning algorithm. The application can be widely applied to the technical field of unmanned aerial vehicle path planning.

Description

Technical Field

[0001] This application belongs to the field of drone path planning technology, and more specifically, it relates to a drone path planning method and system based on 5G signal coverage. Background Technology

[0002] 5G communication drones refer to drones that use 5G technology to interact with the outside world, thereby achieving high-bandwidth, low-latency, and high-stability communication. On the one hand, relying on the high bandwidth, low latency, and high stability transmission characteristics of 5G networks, drones can achieve real-time, low-latency backhaul of large amounts of information such as high-bitrate video streams and multi-dimensional sensor data, providing efficient support for backend data processing and decision-making. On the other hand, with the edge computing architecture and collaborative computing capabilities enabled by 5G technology, the implementation of computing-intensive applications such as drone autopilot, dynamic path planning in complex environments, and multi-drone collaborative operations becomes possible. In summary, the deep integration of 5G technology and drones not only effectively compensates for the shortcomings of insufficient onboard computing power in drones, but also expands the application boundaries of drones in fields such as emergency rescue, environmental monitoring, and smart transportation, which has important theoretical value and practical significance for promoting the development of drone technology towards higher intelligence and wider application scenarios.

[0003] However, drones often experience dynamic obstruction and interference during flight due to environmental factors such as urban buildings and undulating terrain. An existing Chinese invention patent, CN118915814A, provides a method for predicting user trajectories and planning drone paths in real-world geographical environments. While it can indirectly correlate communication performance by calculating path loss and construct a continuous 3D drone path that aligns with user movement trends, this patent only plans paths based on throughput and fails to perceive or predict signal quality. Furthermore, some frequency bands used by 5G communication systems inherently have weak coverage capabilities, and electromagnetic waves are easily obstructed and attenuated by obstacles such as buildings and trees during propagation. Simultaneously, current 5G base station antenna deployments are largely oriented towards ground terminal communication needs, with the antenna main lobe generally facing the ground, resulting in shortcomings in airspace signal coverage. This makes the patent's path planning scheme unable to avoid signal risks. When a drone flies into a signal blind spot or weak area, it can cause communication interruptions and a sudden drop in throughput, leading to weak or even dead zones in the airspace, significantly hindering the stable operation of aerospace applications requiring high bandwidth and low latency. Furthermore, existing drone path planning methods have many shortcomings, such as the lack of real-time perception of 5G signal coverage, leading to entry into signal blind spots; traditional path planning algorithms do not fully consider communication quality constraints; static maps cannot adapt to dynamically changing signal environments; and insufficient signal prediction accuracy makes it difficult to provide a reliable basis for path planning. Therefore, there is an urgent need to provide an intelligent path planning method that can perceive 5G signal coverage in real time, accurately predict signal distribution, and optimize flight paths accordingly to ensure the stable operation of 5G communication drones. Summary of the Invention

[0004] The purpose of this application is to provide a method and system for drone path planning based on 5G signal coverage, so as to solve the technical problem in the prior art that drones cannot dynamically adapt to 5G signals to optimize drone paths during flight, resulting in limited or failed communication rates.

[0005] To achieve the above objectives, the first embodiment of this application provides a method for drone path planning based on 5G signal coverage, comprising the following steps: Data samples are periodically acquired and deep learning algorithms are trained. Based on the trained deep learning algorithms, the predicted values ​​of physical layer signal quality indicators in the environment are continuously predicted. Construct a radio map based on the predicted values ​​of physical layer signal quality indicators; Based on radio maps, a reinforcement learning algorithm is used to plan the flight path of a drone.

[0006] Preferably, the problem of planning the flight path of the UAV is modeled as a Markov decision process, and a reinforcement learning algorithm based on deep deterministic policy gradient is used to optimize the path by constructing a perception module, a decision module and an evaluation module, and to plan the flight path of the UAV.

[0007] Preferably, the Markov decision process includes a state space, an action space, a reward function, state transition probabilities, and a discount factor; The reward function includes path progress reward, signal quality reward, and path smoothness reward.

[0008] Preferably, the process of planning the drone's flight path includes: initializing the drone's starting position, target position, and current state; For each time step, based on the current state, the decision module outputs an action, executes the action, moves the drone to a new location, queries the radio map for the predicted value of the physical layer signal quality index of the new location, constitutes a new state, and completes the state update; The reward for the current step is calculated based on the reward function. If the target position or the maximum number of steps is reached, the flight is terminated and the drone's flight path is output. Otherwise, the decision module continues to output actions until the target position or the maximum number of steps is reached, at which point the flight stops and the drone's flight path is output.

[0009] Preferably, the process of training the deep learning algorithm includes: organizing the data samples according to the time series, constructing a dataset, defining the space and dividing it into several equally divided grids with the position of the UAV at the last moment of the time series as the center, calculating the real signal quality index value of each grid, and using the deep learning algorithm to predict the predicted value of the signal quality index of the physical layer of the grid. The loss function is used to iteratively calculate the loss between the actual signal quality index value and the predicted value of the physical layer signal quality index, and update the network parameters of the deep learning algorithm until the loss function converges or the preset training period is reached, and then training is stopped to obtain the trained deep learning algorithm.

[0010] Preferably, the formula for obtaining the predicted value of the physical layer signal quality index is: ; In the formula, Base stations predicted by the RNN model In grid location RSRP value, and Base stations In grid location Predicted RSRQ and RSSI values.

[0011] Preferably, the process of constructing a radio map includes: dividing the space into three-dimensional grid cells, with each grid cell storing a set of predicted values, update timestamps, and confidence levels for the physical layer signal quality indicators of each base station; Based on the predicted values ​​of the physical layer signal quality indicators, the radio map is updated. For grid cells with existing predicted values, the predicted values ​​of the physical layer signal quality indicators obtained by exponentially weighted fusion of signal data and the existing predicted values ​​are used. The formula is as follows: ; In the formula, The signal data is obtained by exponential weighted average fusion. To integrate the weights, update the timestamp to the current moment. To predict the physical layer signal quality index, These are existing predicted values.

[0012] Preferably, the perception module is used to receive input from the state space and perform standardized processing on the features of each dimension; The decision module is used to output corresponding actions based on the input state characteristics; The evaluation module is used to output the Q-value of the corresponding state-action pair based on the concatenated vector of the state vector and the action vector.

[0013] Preferably, the data samples include physical layer signal quality indicators and UAV location information; Physical layer signal quality metrics include the RSRP, RSRQ, and RSSI values ​​of the current serving base station and neighboring base stations; The location information of a drone includes relative location information and absolute location information; Relative position information includes the UAV in a relative coordinate system. , , The coordinate values; The absolute location information includes the drone's longitude, latitude, and altitude.

[0014] The second embodiment of this application provides a drone path planning system based on 5G signal coverage, including: a preprocessing module, a radio map construction module, and a path planning module; The preprocessing module is used to periodically acquire data samples and train deep learning algorithms. Based on the trained deep learning algorithms, the predicted values ​​of physical layer signal quality indicators in the environment are continuously predicted. The radio map building module is used to build radio maps based on predicted values ​​of physical layer signal quality indicators; The path planning module is used to plan the flight path of the UAV based on the radio map and the reinforcement learning algorithm.

[0015] The beneficial effects of this application are as follows: This application provides a method and system for UAV path planning based on 5G signal coverage. By integrating deep learning algorithms and reinforcement learning algorithms, a closed loop of signal prediction and path planning is achieved. Specifically, firstly, data samples are periodically acquired, and a deep learning algorithm is trained based on the data samples. This enables the algorithm to accurately predict the distribution of 5G signal quality in space under sparse data conditions, thereby constructing a dynamically updated radio map. This provides a reliable basis for UAVs to avoid areas with poor signal and effectively avoids signal blind spots. During the construction of the radio map, the system dynamically updates the radio map based on continuously acquired prediction values, enabling it to quickly respond to environmental changes and improve the robustness and adaptability of the system. Finally, using a reinforcement learning algorithm, with the radio map (including signal quality and positional relationships) generated by the deep learning algorithm as the environmental input, and 5G signal quality as the core of path planning, combined with a continuous reward function, path smoothness constraints, and safety constraint mechanisms, the flight path is optimized while ensuring communication quality. This ensures the stability of the communication link throughout the UAV's flight, achieving a multi-objective balance between signal coverage, path efficiency, and flight stability, and ensuring the safe flight of the UAV in complex environments. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A schematic diagram of the overall process of a drone path planning method based on 5G signal coverage provided in an embodiment of this application; Figure 2 A flowchart of a deep learning training algorithm provided in an embodiment of this application; Figure 3 A flowchart for constructing a complete flight path trajectory is provided for one embodiment of this application. Detailed Implementation

[0018] To make the technical problems, technical solutions, and beneficial effects to be solved by this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this application.

[0019] This application provides a drone path planning method based on 5G signal coverage. It uses deep learning algorithms to predict 5G signals and reinforcement learning algorithms to optimize the path. By integrating signal prediction perception and optimized path planning, it achieves optimal path planning for drones within 5G signal coverage.

[0020] Please see Figure 1 The first embodiment of this application provides a drone path planning method based on 5G signal coverage, comprising: S1: Periodically acquire data samples and train a deep learning algorithm, and continuously predict the predicted values ​​of physical layer signal quality indicators in the environment based on the trained deep learning algorithm.

[0021] During the flight of the drone, a period of The periodic data acquisition method obtains physical layer signal quality indicators and UAV position information in real time, which are used to form data samples, represented as follows: ; In the formula, For data samples, In order to be in At any moment, drones and base stations Physical layer signal quality metrics between In order to be in At any given moment, the relative position information of the drone. In order to be in At any given moment, the absolute position information of the drone. For the time of data collection, For time sets, For base stations, The set of base stations within the visible range. .

[0022] Physical layer signal quality metrics characterize the 5G signal strength and quality at the drone's current location, including the RSRP (Reference Signal Received Power), RSRQ (Reference Signal Received Quality), and RSSI (Received Signal Strength Indicator) of the current serving base station and neighboring base stations. For At any given moment, drones and a single base station Physical layer signal quality indicators Represented as: ; In the formula, 、 、 They represent in At any time, base station RSRP value, RSRQ value, RSSI value.

[0023] The drone's location information includes relative location information and absolute location information, among which, At any given time, the relative position information of the drone is represented as follows: ; In the formula, , , They represent drones In relative coordinate system at all times , , The coordinate values; exist At any given moment, the absolute position information of the drone is represented as follows: ; In the formula, , , These represent the longitude, latitude, and altitude of the drone, respectively.

[0024] In one optional embodiment, the deep learning algorithm employs a recurrent neural network (RNN) model to predict the signal distribution in the space surrounding the drone using sparse measurement data along the drone's flight path for a single base station. The type of deep learning algorithm is not limited here and can be selected according to the actual situation.

[0025] Specifically, please refer to Figure 2 First, the RNN model is trained. The training process includes: constructing a training dataset, including a training set and a validation set; and organizing the collected periodic data samples according to time series for a single base station. Each training sample contains continuous The data at each time point is represented as follows: ; In the formula, For the training sample set, The time window length can be set according to the device's computing power and the expected model size. It is a 9-dimensional vector containing 3-dimensional physical layer signal quality indicators and 6-dimensional UAV position information. Next, taking the input time series as the last moment and the drone's position as the center, the length, width, and height are defined as follows: , , A rectangular space with a unit distance, the actual physical layer signal quality index value of each grid point in this space. Represented as: ; ; ; ; In the formula, For base stations In grid location RSRP value, and Base stations In grid location The RSRQ and RSSI values.

[0026] The predicted values ​​of the physical layer signal quality indicators for each grid point within the aforementioned space, as predicted by the RNN model. , is represented as: ; ; ; ; In the formula, Base stations predicted by the RNN model In grid location RSRP value, and The base stations predicted by the RNN model are respectively In grid location Predicted RSRQ and RSSI values.

[0027] To evaluate the accuracy of the RNN model's predictions, a loss function needs to be further defined. This application does not impose restrictions on the choice of loss function; it can be selected according to the actual situation.

[0028] In an optional embodiment, a mean squared error loss function is employed. Iteratively calculate the true physical layer signal quality index values ​​for all training samples. The predicted values ​​of the physical layer signal quality index obtained by the RNN model The smaller the loss between the two, the higher the prediction accuracy of the RNN model, which is expressed as: ; In the formula, The number of training samples. For the first The true label corresponding to each training sample For the RNN model, the first The predicted output of each training sample.

[0029] Finally, the network parameters of the deep learning algorithm are updated using the backpropagation algorithm and the Adam optimizer, wherein the learning rate of the Adam optimizer is set to... To prevent overfitting, a dropout rate is added to the deep learning algorithm. The Dropout layer is used, and L2 regularization is applied to constrain the weight parameters. During the training of the deep learning algorithm, an early stopping strategy is adopted. If the loss on the validation set fails to improve for several consecutive rounds, the algorithm is terminated early. If the early stopping strategy is not triggered, the deep learning algorithm continues to train until the loss function converges or the preset number of training rounds is reached, at which point training stops, resulting in the trained deep learning algorithm (RNN model).

[0030] Using a trained deep learning algorithm (RNN model), the current Time and before Using the data at each time point as input, the output is the predicted value of the physical layer signal quality index. .

[0031] S2: Construct a radio map based on the predicted values ​​of the physical layer signal quality indicators.

[0032] Predicted values ​​of physical layer signal quality indicators based on continuous periodicity prediction The radio map is used to record, integrate, and query predicted values ​​of physical layer signal quality indicators at various locations in space, thereby achieving a unified representation of the global signal environment. Specific functions include querying predicted values, integrating predicted values, and maintaining consistency.

[0033] Radio maps are stored using a three-dimensional grid structure, dividing the space into three-dimensional grid cells. The grid resolution can be set according to actual needs, and each grid cell synchronously stores the base stations within that cell. The set of predicted values ​​for signal quality indicators, their corresponding update timestamps, and confidence levels are represented as follows: ; In the formula, The data structure stored for each grid cell, This is the set of predicted physical layer signal quality metrics for all visible base stations at this location. This refers to the spatial location information corresponding to this grid cell. This is the update timestamp for the predicted signal quality index value in this grid cell. This represents the confidence level of the predicted signal quality index value in this grid cell.

[0034] When it is necessary to query the predicted value of physical layer signal quality indicators based on a radio map, input the UAV's location information (relative location information). or absolute location information The corresponding grid position is obtained through coordinate transformation. The predicted set of physical layer signal quality indices for all visible base stations at that location is retrieved from the radio map. ,in Includes base stations The predicted values ​​of RSRP, RSRQ, and RSSI at this location.

[0035] When it is necessary to integrate predicted values ​​or dynamic updates of physical layer signal quality indicators, the input should be specific to the base station. The predicted values ​​of the physical layer signal quality indexes predicted by the model The corresponding base station in the radio map The predicted values ​​of the physical layer signal quality metrics at the corresponding grid locations are updated as follows: The new predicted values ​​are dynamically updated. For grids with existing predicted values, an exponentially weighted average is used to fuse the signal data, as shown in the formula: ; In the formula, The signal data is obtained by exponential weighted average fusion. To integrate the weights, update the timestamp to the current moment. For the predicted signal data, These are existing predicted values.

[0036] If multiple base stations Since the prediction regions overlap, a distance-weighted averaging method is used to fuse the predicted values ​​of physical layer signal quality indicators within the overlapping regions. The weights are determined by combining the freshness of the prediction time and the prediction confidence level to ensure the consistency of the global data. The formula for the distance-weighted averaging is: ; In the formula, This is the predicted value of the final physical layer signal quality index after fusion at this location. , From grid point to base station distance, For base stations At this location The predicted value of the physical layer signal quality index at that location.

[0037] S3: Based on the radio map, a reinforcement learning algorithm is used to plan the drone's flight path.

[0038] The problem of planning the flight path of a UAV is modeled as a Markov decision process. A reinforcement learning algorithm based on deep deterministic policy gradient (DDPG) is used to optimize the path by constructing a perception module, a decision module, and an evaluation module to plan the UAV flight path.

[0039] Specifically, the UAV path planning problem is modeled as a Markov Decision Process (MDP), including the state space. Action space Reward function State transition probability and discount factor .

[0040] state space Including intelligent agents Status information at any time , is represented as: ; In the formula, for The absolute position of the drone at all times. for The absolute position of the drone in real time is used to reflect its direction and speed of movement. The current velocity vector, For drones The change in displacement along the x-axis at time t. For drones The change in displacement along the y-axis at time t. For drones The change in displacement along the z-axis at time t. For base station set China's base stations exist The signal characteristics at any given time can be obtained by querying radio maps. These are the absolute coordinates of the target location. for Remaining battery percentage at any time for The remaining Euclidean distance from the current position to the target position is calculated using the following formula: ; In the formula, for The absolute position of a moment These are the absolute coordinates of the target location.

[0041] Action space Let be the continuous action space, representing the three-dimensional position adjustment vector that the agent can execute, denoted as: ; In the formula, To adjust the vector, , , They are respectively , , The amount of directional position adjustment. To adjust the magnitude of the Euclidean distance of the vector, and These are the minimum and maximum step sizes, set according to the drone's motion performance and mission requirements.

[0042] reward function Rewards based on path progress Signal quality reward and path smoothness reward It consists of three parts, represented as: ; In the formula, As a reward for path progress, As a reward for signal quality, As a reward for path smoothness, and These are the corresponding weighting coefficients.

[0043] Furthermore, path progress rewards The approach of the drone to the target point is determined and expressed as follows: ; In the formula, when the drone approaches the target point, It is a positive value; when moving away from the target point, It is a negative value.

[0044] Signal quality reward A continuous reward function is used to provide different levels of reward based on the signal quality index strength at the current location, expressed as: ; In the formula, , , These are the weighting coefficients for each signal indicator; , , The function maps signal strength to reward value, employing either a piecewise linear function or a sigmoid function, such that stronger signals receive higher rewards, while signals below a threshold receive negative rewards. , and Each is the current The RSRP, RSRQ, and RSSI values ​​at the location of the drone at any given time.

[0045] Path smoothness reward Used to penalize frequent changes of direction and improve flight stability, it is represented as: ; In the formula, This is the smoothness penalty coefficient. The penalty is the difference between two adjacent actions; the greater the difference, the greater the penalty.

[0046] State transition probability The dynamic model of the drone determines, under ideal conditions, the execution of actions. Afterwards, the drone's position is definitively transferred to the new state, as shown in the formula: ; In the formula, for The absolute position of the drone at all times. for The absolute position of the drone at all times. To adjust the vector.

[0047] Furthermore, discount factor Used to balance immediate and long-term rewards, with a value range of [value range missing]. The preferred value is .

[0048] The Deep Deterministic Policy Gradient (DDPG) reinforcement learning algorithm employs an Actor-Critic architecture, collaboratively optimizing paths by constructing a perception module, a decision module, and an evaluation module. The perception module receives the state space. The input is used to standardize the features of each dimension. The standardization formula is as follows: ; In the formula, The state vector of the th One portion, and The first The mean and standard deviation of each component are obtained through historical data statistics. After standardization, the output is a feature vector adapted to the input of the neural network. .

[0049] The decision module (Actor network) is constructed using a multi-layer fully connected neural network. The network structure includes an input layer, several hidden layers, and an output layer. The ReLU activation function is used between each layer, and the input is standardized state features. The action that outputs a continuous three-dimensional position adjustment vector is given by the following formula: ; In the formula, For action, , They are respectively , , The amount of three-dimensional position adjustment in direction.

[0050] To ensure exploratory learning, the output movement is performed during the training phase. The Ornstein-Uhlenbeck (OU) process noise or Gaussian noise is superimposed on top, and the formula is: ; In the formula, For actions after adding noise, The action selected for the agent. For the noise term, OU noise can be used, and the formula is: ; In the formula, This is the noise term from the previous moment. For the regression speed, This is the long-term mean. For time step, To adjust the noise intensity, attenuated Gaussian noise can also be used. , It gradually diminishes during the training process.

[0051] The evaluation module (Critic network) is constructed using a multi-layer fully connected neural network, with state features as input. Actions output by the decision module spliced ​​vector The output is the state-action pair. Q The value is represented as: ; Should Q Value evaluation in state Next action The expected long-term cumulative reward.

[0052] Furthermore, the Actor and Critic networks are updated, and the update process is as follows: First, the Critic network is updated using the temporal difference method, with the loss function... for: ; In the formula, For the goal Qvalue, The reward for the i-th sample. As a discount factor, For batch size, For state and actions of Q value, and These are the outputs of the target Critic network and the target Actor network, respectively.

[0053] Next, the Actor network is updated by optimizing the policy by maximizing the Q-value, using the policy gradient method to update the Actor network parameters. Objective function for: ; In the formula, Let Actor be the objective function of the Actor network. For the Critic network to the Actor network in state The value score of the generated action. This refers to the batch size.

[0054] Finally, update the target network parameters using a soft update method: ; In the formula, For the current network parameters, For the target network parameters, This is the soft update coefficient, and its value range is... The preferred value is .

[0055] Furthermore, during the reinforcement learning and training phase, an experience playback mechanism is used to improve training stability and efficiency.

[0056] Specifically, an experience replay buffer is first constructed to store historical experience samples. ,in, This is the current state. In order to perform the action, In order to obtain the reward, This is the state after the transition.

[0057] During training, a batch of empirical samples are randomly sampled from the buffer, and the network parameters are updated based on the batch of empirical samples. This breaks the correlation between sample data, avoids the interference of data sequence in the training process, and improves training stability.

[0058] The Prioritized Experience Replay (PER) mechanism is adopted, and training is based on temporal difference error (TD-error). Assign sampling priorities to empirical samples, where, For the goal Q value, For the current output of the Critic network Q During training, samples with larger TD errors are sampled first, so that the model focuses more on experiences with high errors, thus accelerating the learning and convergence process.

[0059] In an alternative embodiment, the reinforcement learning algorithm adds safety constraint mechanisms, including boundary constraints, collision avoidance, and energy management.

[0060] Specifically, the flight range of the drone is constrained, limiting its flight range to a preset three-dimensional spatial boundary. Within the bounds, when a predicted action leads to an out-of-bounds situation, the action is either truncated or a large negative reward is given to fulfill the boundary constraints.

[0061] Obstacle distance information is introduced into the state space, and a collision penalty term is added to the reward function. ,in, The distance from the current position to the nearest obstacle. For safe distance threshold, This is a penalty coefficient used to achieve collision avoidance.

[0062] Monitor remaining battery power When the battery level is below the safety threshold, a return-to-home reward is added or the target point is changed to the starting position to ensure that the drone can return safely, thus achieving energy management.

[0063] When the preset maximum number of training rounds is reached, or continuously The average increase in reward over the evaluation period is less than the preset threshold. This terminates the training of the reinforcement learning algorithm. Here, we... and There are no restrictions; settings can be made according to actual application needs.

[0064] In practical applications, reinforcement learning algorithms perform path planning according to the following process: Specifically, please refer to Figure 3 First, initialize the system and set the drone's starting position. and target location Initialize the current state At each time step According to the current state The decision module outputs actions. Execute the action The drone moves to a new location and queries the radio map for the predicted values ​​of the physical layer signal quality indicators for that location, thus establishing a new state. Status update complete.

[0065] Then, based on the reward function Calculate the reward for the current step If the target location is reached ( , If the target threshold is reached or the maximum number of steps is reached, the flight will terminate and the complete flight path trajectory will be output. Otherwise, the decision module continues to output actions. The drone will stop flying once it reaches the target location or the maximum number of steps, and output its flight path.

[0066] The second embodiment of this application provides a drone path planning system based on 5G signal coverage, including: The preprocessing module is used to periodically acquire data samples and train deep learning algorithms. Based on the trained deep learning algorithms, the predicted values ​​of physical layer signal quality indicators in the environment are continuously predicted.

[0067] The radio map building module is used to build radio maps based on predicted values ​​of physical layer signal quality indicators.

[0068] The path planning module is used to plan the flight path of the UAV based on the radio map and a reinforcement learning algorithm.

[0069] Example 1: Drone delivery experiment in an urban environment.

[0070] In an urban environment, a 5G base station network has been deployed in a certain area, with a coverage area of ​​[area missing]. The area contains multiple 5G base stations. The drone is equipped with a 5G communication module, GPS positioning system, and onboard computing unit, and uses a quadcopter structure with a maximum flight speed of [missing information]. The battery life is approximately 30 minutes.

[0071] The system parameters are set as follows: Data collection cycle: Second; Time window length ; Radio map: Grid resolution set to ; Reinforcement learning parameters: discount factor Learning rate ; Action space: minimum step size meters, maximum step size rice; Signal threshold: , , ; When the drone performs its first flight mission, it follows a preset data collection cycle. The system periodically records the 5G signal quality indicators for the current location every second. At any given moment, the collected data samples include physical layer signal quality indicators and UAV location information.

[0072] Specifically, regarding the current serving base station and three neighboring base stations (assuming...) The RSRP, RSRQ, and RSSI values ​​are measured respectively to form the physical layer signal quality index. Obtain absolute location via GPS ,in, For longitude, For latitude, Altitude; relative position obtained through inertial navigation system. By combining the absolute and relative positions of the drone, the drone's position information is obtained.

[0073] In this embodiment, At what time, the data collected for base station 1 is as follows: ; In the formula, For base station 1 at time Data collected per second This is the physical layer signal quality vector, with RSRP value of -95dB, RSRQ value of -12dB, and RSSI value of -75dB. This is a relative position vector, with the x-axis coordinate being 50 meters, the y-axis coordinate being 30 meters, and the z-axis coordinate being 100 meters. These are absolute position coordinates. Longitude Latitude This refers to altitude.

[0074] After obtaining the data samples, the deep learning algorithm is trained based on the data samples, and the predicted values ​​of the physical layer signal quality index are continuously predicted.

[0075] First, a training dataset is constructed. Historically collected data is organized by time series, targeting individual base stations. Each training sample contains continuous The data was collected at each time point. The dataset was divided into training, validation, and test sets in a 7:2:1 ratio.

[0076] Then, a deep learning algorithm is constructed. In this embodiment, an LSTM model is used. The LSTM model is constructed using the PyTorch framework, and the network structure is as follows: Input layer (9 dimensions) → LSTM layer 1 (128 hidden units) → Dropout layer (dropout rate 0.3) → LSTM layer 2 (128 hidden units) → Fully connected layer → Output layer ( Dimension, among which Corresponding prediction Three signal indicators for the grid.

[0077] Next, the model is trained using the Adam optimizer, with the learning rate set to... The batch size is 64, and the training lasts for 200 epochs. The loss function used is mean squared error (MSE), and the best MSE loss on the validation set during training is... An early stopping strategy is adopted, which terminates training prematurely when the validation set loss does not improve for 10 consecutive epochs.

[0078] Finally, the predicted values ​​of the physical layer signal quality index are calculated. After training, during real-time flight, data from the current time and the past nine time points are input into the LSTM model, which outputs the predicted values ​​of the physical layer signal quality index. , including the surrounding Predicted values ​​of RSRP, RSRQ, and RSSI at each point within the grid space.

[0079] A radio map is constructed based on the predicted values ​​of the physical layer signal quality indicators obtained through continuous prediction.

[0080] First, create a 3D mesh-structured radio map, covering an area of... rice, rice, meters, grid resolution is ,total There are 1 grid cell. Each grid cell stores data in the following structure: ; Next, the predicted values ​​are integrated. Each time, the LSTM model outputs a predicted value for the physical layer signal quality index. Then, the predicted values ​​are written to the corresponding grid positions. For grids with existing predicted values, an exponentially weighted average is used for merging: ; In the formula, To integrate the weights, update the timestamp to the current moment.

[0081] Finally, consistency maintenance is performed. When the predicted regions of multiple base stations overlap, the predicted values ​​of the physical layer signal quality index within the overlapping region are fused using a distance-based weighted average. The formula is as follows: ; In the formula, This is the predicted value of the final physical layer signal quality index after fusion at this location. , From grid point to base station distance, For base stations At this location The predicted value of the physical layer signal quality index at that location.

[0082] After constructing the radio map, reinforcement learning algorithms are used to plan the drone's flight path based on the radio map.

[0083] Specifically, first, set the environment and the starting point. (Delivery origin), destination (Delivery destination).

[0084] Then initialize the DDPG network: For the Actor network, the input dimension is the state space dimension (approximately 20 dimensions, including position, velocity, signal indicators, etc.), and the hidden layer structure is as follows. The output is a 3D continuous action (3D position adjustment vector); for the Critic network, the input is a concatenation of state and action (approximately 23 dimensions), and the hidden layer structure is as follows. Output a 1D Q-value; simultaneously create target networks for the Actor and Critic, respectively, and set the soft update coefficients of the target networks. Additionally, an experience replay buffer is constructed, with its capacity set to 100,000 experiences.

[0085] Next, the flight range of the drone is constrained, and the flight boundary is set as follows: rice, rice, Meters. Simultaneously, five static obstacles (buildings) are placed in the environment, with a safe distance threshold. Meters. Set the drone's battery capacity to 8000mAh and the safe return-to-home threshold to 30%.

[0086] After initializing the DDPG network, 1000 training rounds are conducted, each round including the following steps: First, initialize the state. It executes a decision loop of up to 200 steps, with each step based on the action output by the Actor network. Superimposed OU noise ( , After the drone virtually moves to a new location, it queries the current signal strength from the radio map to calculate the reward. (wherein the path progress reward weight) Signal quality reward weight Smoothness penalty coefficient ). Experience The experience is stored in the experience replay buffer, and 64 experiences are sampled from the experience replay buffer to update the Critic and Actor networks, thereby achieving a soft update of the target network parameters.

[0087] In actual flight, the drone adjusts its flight path based on the current status. The trained Actor network outputs deterministic actions. It then performs position adjustments to plan the drone's flight path.

[0088] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0089] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for unmanned aerial vehicle (UAV) path planning based on 5G signal coverage, characterized in that, Includes the following steps: Data samples are periodically acquired and a deep learning algorithm is trained. Based on the trained deep learning algorithm, the predicted values ​​of physical layer signal quality indicators in the environment are continuously predicted. A radio map is constructed based on the predicted values ​​of the physical layer signal quality indicators; Based on the aforementioned radio map, a reinforcement learning algorithm is used to plan the UAV's flight path.

2. The drone path planning method based on 5G signal coverage as described in claim 1, characterized in that, The problem of planning the flight path of the UAV is modeled as a Markov decision process. A reinforcement learning algorithm based on deep deterministic policy gradient is used to optimize the path by constructing a perception module, a decision module, and an evaluation module, and to plan the flight path of the UAV.

3. The drone path planning method based on 5G signal coverage as described in claim 2, characterized in that, The Markov decision process includes a state space, an action space, a reward function, state transition probabilities, and a discount factor. The reward function includes path progress reward, signal quality reward, and path smoothness reward.

4. The drone path planning method based on 5G signal coverage as described in claim 3, characterized in that, The process of planning the drone's flight path includes: initializing the drone's starting position, target position, and current state; For each time step, based on the current state, the decision module outputs an action, executes the action, moves the UAV to a new location, queries the radio map for the predicted value of the physical layer signal quality index of the new location, constitutes a new state, and completes the state update; The reward for the current step is calculated based on the reward function. If the target position or the maximum number of steps is reached, the flight is terminated and the drone flight path is output. Otherwise, the decision module continues to output actions until the target position or the maximum number of steps is reached, at which point the flight stops and the drone flight path is output.

5. The drone path planning method based on 5G signal coverage as described in claim 1, characterized in that, The process of training the deep learning algorithm includes: organizing the data samples according to the time series, constructing a dataset, defining the space and dividing it into several equally divided grids with the position of the UAV at the last moment of the time series as the center, calculating the real signal quality index value of each grid, and using the deep learning algorithm to predict the predicted value of the physical layer signal quality index of the grid. The loss function is used to iteratively calculate the loss between the actual signal quality index value and the predicted value of the physical layer signal quality index, and update the network parameters of the deep learning algorithm until the loss function converges or the preset training period is reached, and then training is stopped to obtain the trained deep learning algorithm.

6. The drone path planning method based on 5G signal coverage as described in claim 1, characterized in that, The formula for obtaining the predicted value of the physical layer signal quality index is as follows: ； In the formula, Base stations predicted by the RNN model In grid location RSRP value, and Base stations In grid location Predicted RSRQ and RSSI values.

7. The drone path planning method based on 5G signal coverage as described in claim 5, characterized in that, The process of constructing the radio map includes: dividing the space into three-dimensional grid cells, with each grid cell storing a set of predicted values, update timestamps, and confidence levels of the physical layer signal quality indicators of each base station; Based on the predicted values ​​of the physical layer signal quality index, the radio map is updated. For grid cells with existing predicted values, the predicted values ​​of the physical layer signal quality index obtained by exponentially weighted fusion of signal data and the existing predicted values ​​are used, with the following formula: ； In the formula, The signal data is obtained by exponential weighted average fusion. To integrate the weights, update the timestamp to the current moment. To predict the physical layer signal quality index, These are existing predicted values.

8. The drone path planning method based on 5G signal coverage as described in claim 4, characterized in that, The perception module is used to receive input from the state space and perform standardized processing on the features of each dimension. The decision module is used to output the corresponding action based on the input state characteristics; The evaluation module is used to output the Q value of the corresponding state-action pair based on the concatenation vector of the state vector and the action vector.

9. The drone path planning method based on 5G signal coverage as described in claim 1, characterized in that, The data samples include physical layer signal quality indicators and UAV location information; The physical layer signal quality indicators include the RSRP value, RSRQ value, and RSSI value of the current serving base station and neighboring base stations; The drone's location information includes relative location information and absolute location information; The relative position information includes the UAV's position in a relative coordinate system. , , The coordinate values; The absolute position information includes the longitude, latitude, and altitude of the UAV.

10. A drone path planning system based on 5G signal coverage, applied to the drone path planning method based on 5G signal coverage as described in any one of claims 1-9, characterized in that, include: Preprocessing module, radio map construction module, and route planning module; The preprocessing module is used to periodically acquire data samples and train a deep learning algorithm, and based on the trained deep learning algorithm, continuously predict the predicted values ​​of physical layer signal quality indicators in the environment. The radio map building module is used to build a radio map based on the predicted values ​​of the physical layer signal quality indicators; The path planning module is used to plan the flight path of the UAV based on the radio map and employing a reinforcement learning algorithm.

Images