Three-dimensional path planning method for unmanned aerial vehicle based on artificial potential field enhancement and reinforcement learning
By constructing a dual-stream collaborative perception network and combining artificial potential fields and reinforcement learning, the path planning problem of UAVs in complex environments is solved, achieving efficient obstacle avoidance and accurate arrival, which is applicable to small civilian UAVs.
Patent Information
- Application Number
- CN202611175724.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-04
- Publication Date
- 2026-11-03
AI Technical Summary
Existing UAV path planning algorithms suffer from slow training convergence, easy obstacle collisions during initial random exploration in complex 3D environments, easy getting trapped in local minima using traditional artificial potential field methods, and inability to adapt to lightweight airborne computing power. Furthermore, they cannot meet the safe and efficient flight requirements of civilian UAVs in complex environments.
By employing a method based on artificial potential field enhancement and reinforcement learning, a dual-stream collaborative perception network is constructed. Features are extracted in parallel through terrain perception stream and potential field perception stream, and path planning is optimized by combining the PPO algorithm, enabling UAVs to achieve efficient obstacle avoidance and accurate arrival in complex environments.
It significantly improves the obstacle avoidance accuracy of drones in complex mountainous and urban environments, reduces the probability of fuselage collision damage, shortens training convergence time, and improves training efficiency and stability. It is also compatible with lightweight onboard computing platforms for small civilian drones.
Smart Images
Figure CN123028595A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of control or regulation of non-electrical variables, and more specifically to a method for three-dimensional path planning of unmanned aerial vehicles based on artificial potential field enhancement and reinforcement learning. Background Technology
[0002] With the rapid popularization of intelligent and fully autonomous flight technology in civilian drones, they are being used on a large scale in scenarios such as mountain earthquake emergency search and rescue, complex mountain terrain mapping, urban last-mile logistics delivery, mountain power line inspection, and agricultural and forestry plant protection aerial photography. These civilian scenarios all involve complex static and dynamic obstacles such as undulating mountains, high-rise buildings, high-voltage power lines, pedestrians, and mobile work vehicles. Drones need to autonomously generate safe flight paths in a high-dimensional, dynamic, and multi-obstacle three-dimensional space. Path planning must simultaneously meet the drone's own dynamic constraints, such as pitch angle and minimum turning radius, as well as multiple constraints such as geometric obstacle avoidance, endurance and energy saving, and flight safety. Therefore, achieving multi-constraint, real-time, and safe global three-dimensional path planning under the condition of lightweight onboard computing power is a key technical challenge that urgently needs to be solved in the field of civilian drone control.
[0003] Existing path planning methods have significant limitations: Artificial Potential Field (APF) has a simple mathematical model, low computational cost, and good airborne real-time performance, but it is prone to getting stuck in local minima in complex 3D environments such as continuous mountainous terrain and urban buildings, and suffers from the defect of unreachable target points. It is difficult to accurately characterize undulating terrain and cannot predict and avoid dynamic interference sources such as moving pedestrians and vehicles. Proximal Policy Optimization (PPO) algorithm in Deep Reinforcement Learning (DRL) does not require accurate environment modeling and has excellent generalization performance, but it has defects when directly applied to 3D continuous space planning for civilian UAVs: the huge state-action space dimension causes slow training convergence, the sparse reward leads to training oscillation and degradation during long-distance inspection and navigation, the irregular exploration in the early stage of training is prone to collisions with mountains / buildings, causing equipment damage, and the learning of abstract features of terrain and obstacle potential fields requires massive training samples, resulting in insufficient robustness of airborne deployment.
[0004] Current solutions that combine artificial potential fields with deep reinforcement learning only perform simple numerical superposition and action-level fusion, failing to achieve collaborative fusion at the feature level and losing the complete safety guidance information contained in the spatial distribution of the potential field. Real-time calculation of the global potential field faces a contradiction between computing power and accuracy. The onboard computing power of civilian small UAVs is limited, making it difficult to generate high-resolution global potential field distribution maps in real time. At the same time, existing solutions do not distinguish between two types of heterogeneous environmental information: terrain obstacles and potential field safety guidance. The network is prone to environmental misjudgment and cannot meet the needs of safe, efficient, and low-energy autonomous flight of UAVs in complex three-dimensional environments in civilian applications. Summary of the Invention
[0005] The purpose of this invention is to provide a UAV 3D path planning method based on artificial potential field enhancement and reinforcement learning. By constructing a dual-stream collaborative perception network, the artificial potential field is transformed into a visual feature map to assist reinforcement learning decision-making, achieving efficient obstacle avoidance and accurate arrival in complex environments. This addresses the problems of slow training convergence, easy obstacle collision during initial random exploration, susceptibility to local minima in traditional artificial potential field methods, and inability to adapt to lightweight airborne computing power in existing civilian UAV path planning algorithms.
[0006] To achieve the above objectives, the present invention adopts the following technical solution: The UAV 3D path planning method based on artificial potential field enhancement and reinforcement learning includes the following steps: S1. Construct a three-dimensional flight simulation environment, and define a UAV multimodal observation space based on the three-dimensional flight simulation environment, which includes a local topographic map, a local artificial potential field map to be generated, and flight state vectors; S2. Based on the multimodal observation space, map the three-dimensional terrain data around the current location of the UAV into a two-dimensional grid matrix and complete the normalization process; S3. Construct a local observation window centered on the current two-dimensional coordinates of the UAV and matching the size of the local topographic map, map and generate a global coordinate matrix, calculate the gravitational potential energy matrix and repulsive potential energy matrix in parallel, combine the boundary mask to generate a local total potential field matrix and supplement it to the multimodal observation space. S4. Construct a dual-stream collaborative sensing network containing terrain sensing flow and potential field sensing flow, and input the local terrain map and local total potential field matrix into the dual-stream collaborative sensing network to extract terrain geometric texture and physical boundary feature vectors, environmental hazard gradient and virtual safety passage feature vectors respectively; S5. The two types of feature vectors extracted in step S4 are concatenated with the flight state vector to form a joint feature vector, which is then input into the PPO algorithm backend network to output the action probability distribution corresponding to the UAV's discrete flight actions. The UAV performs the corresponding flight actions according to the action probability distribution. S6. Calculate the reward value based on the UAV's flight status, action execution results, and local total potential field matrix, update the parameters of the PPO algorithm backend network and the dual-stream collaborative sensing network, and iteratively optimize the path planning strategy.
[0007] Preferably, the three-dimensional flight simulation environment in step S1 includes a terrain model, fixed target points, and static obstacles.
[0008] Preferably, the local terrain map in step S1 is a local altitude matrix centered on the current position of the UAV, and multiple consecutive simulation steps are stacked to preserve temporal terrain features; the flight state vector includes the three-dimensional coordinates of the UAV in the normalized map, the Euclidean distance to the target point, the remaining fuel amount, and the current time step.
[0009] Preferably, the normalization process in step S2 involves mapping the terrain height from physical space to a dimensionless interval and using a terrain height heatmap as an environmental representation method.
[0010] Preferably, the global coordinate matrix in step S3 is calculated as follows:
[0011] Let the two-dimensional coordinates of the UAV at the current moment be . The index matrix of the local mesh is and Window radius is The global coordinate matrix is and .
[0012] Preferably, the calculation process for the gravitational potential energy matrix in step S3 is as follows: First, utilize the matrix broadcasting property to broadcast the global coordinate matrix. and The square matrix of the Euclidean distance is obtained by performing calculations with the global two-dimensional coordinates of the target point. :
[0013] Substitute the squared matrix of the Euclidean distance into the gravitational potential energy matrix:
[0014] in, The target gravitational decay coefficient, used to control the rate of decay of gravitational potential energy, is a fixed parameter, ultimately yielding the gravitational potential energy matrix. ; The calculation process for the repulsive potential energy matrix is as follows: First, iterate through all threat sources in the environment, based on the global coordinate matrix. and Parallel computation from local mesh to various threat sources Distance matrix : Substitute the distance matrix into the repulsive potential energy matrix: in, For the environment The repulsive field attenuation coefficient of a threat source is a fixed parameter, which can be set to a larger value for high-risk threat sources. By triggering the evasion action in advance, the repulsive potential energy matrix is ultimately obtained. .
[0015] Preferably, the local total potential field matrix mentioned in step S3 The calculation formula is:
[0016] in, For the boundary mask, when grid points When outside the physical boundaries of the map Set the value to 0, and set it to 1 if it does not exceed the boundary. The values at the grid points corresponding to the repulsive potential energy matrix are... The values at the grid points corresponding to the gravitational potential energy matrix are denoted as .
[0017] Preferably, in step S4, the terrain-aware stream and the potential field-aware stream extract features synchronously using parallel computing. Both extract basic features through convolutional layers, and after downsampling to compress the data dimension, a high-level abstract feature vector is output by a fully connected layer. The feature vector output by the terrain-aware stream represents the physical boundary and traversability information of the terrain, while the feature vector output by the potential field-aware stream represents the dangerous gradient distribution and optimal navigation direction of the environment.
[0018] Preferably, the composite reward function in step S6 The specific expression is: Among them, the core reward items , This represents the local total potential field value. Distance Guide Rewards , The distance to the target point at the current moment. The distance from the previous moment. This is the scaling factor; Rewards for completing the task. As a penalty for collision, Penalty for time step; High energy consumption penalty , The current altitude. The altitude at the previous moment, This is the scaling factor.
[0019] Preferably, the parameter update in step S6 is implemented as follows: the parameters of the PPO backend network and the dual-stream collaborative sensing network are bound into a unified parameter set. Based on the reward value calculated by batch empirical data and composite reward function, the unified parameter set is synchronously calculated and updated by an adaptive optimization algorithm. The training ends according to the preset total number of training steps or the convergence of policy performance, and the optimal path planning policy is output.
[0020] By adopting the above technical solution, the present invention has the following advantages compared with the prior art: 1. This invention provides a UAV 3D path planning method based on artificial potential field enhancement and reinforcement learning. The dual-stream collaborative perception network extracts features in parallel through terrain perception stream and potential field perception stream, accurately capturing terrain physical boundaries and potential field danger gradient information respectively, realizing the organic unity of physical hard constraints and logical soft guidance. This enables the UAV to clearly identify insurmountable physical obstacles and predict safe and efficient flight directions, significantly improving the obstacle avoidance accuracy of UAVs in complex civilian scenarios such as mountains and cities, and reducing the probability of fuselage collision damage.
[0021] 2. This invention provides a UAV 3D path planning method based on artificial potential field enhancement and reinforcement learning. The potential energy-driven composite reward function directly integrates the local total potential field value into the reward signal, transforming the sparse reward problem into a dense reward problem. This provides clear guidance for the agent in the early stages of training. Combined with a low-entropy training strategy to suppress ineffective random exploration, it significantly shortens the convergence time of the model from random walk to planning an effective path, reduces computational resource consumption, and improves training efficiency and stability.
[0022] 3. This invention provides a UAV 3D path planning method based on artificial potential field enhancement and reinforcement learning. By comprehensively capturing terrain, potential field and UAV state information through multimodal observation space, and combining the decision-making capabilities of the PPO algorithm backend network, the planned path not only meets multiple constraints such as dynamics and geometric obstacle avoidance, but also has smoothness and energy efficiency, achieving the core requirements of efficient obstacle avoidance and accurate arrival at the target point. It is suitable for all types of civilian UAV 3D autonomous flight planning scenarios such as emergency search and rescue, power line inspection, urban logistics, and agricultural and forestry surveying, and can be deployed on a lightweight computing platform on small civilian UAVs. Attached Figure Description
[0023] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation
[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Example
[0025] Please refer to Figure 1 As shown, this invention discloses a UAV 3D path planning method based on artificial potential field enhancement and reinforcement learning, including the following steps: S1. A 3D flight simulation environment is constructed based on the Python Gymnasium simulation framework, with a fixed spatial scale. An undulating mountainous terrain model containing typical landforms such as peaks, gullies, and gentle slopes is generated using a synthetic terrain generation algorithm. Fixed power inspection towers, material delivery points, and surveying and sampling target points are set up. Various static obstacles, including buildings, mountains, and high-voltage towers, are arranged, covering different geometric shapes and sizes. Various dynamic environmental interference sources, such as moving pedestrians, field operation vehicles, and low-altitude birds, are configured. The danger coverage of different interference sources is positively correlated with their risk level. All dynamic interference sources move randomly according to preset trajectories.
[0026] The three-dimensional flight simulation environment defines a multimodal observation space for UAVs, which includes local topographic maps, a local artificial potential field map to be generated, and flight state vectors.
[0027] Local topographic map: Centered on the current position of the UAV, a local height matrix of fixed pixel size is extracted. Each pixel corresponds to a fixed actual spatial range. Multiple consecutive height matrices of simulation step size are stacked to preserve temporal terrain features. The matrix values are normalized terrain elevations.
[0028] Local artificial potential field map: Initially a blank matrix with the same size as the local topographic map, it is subsequently generated and added to the observation space through step S3, and multiple potential energy matrices of simulation step size are stacked.
[0029] Flight state vector: contains normalized data in multiple dimensions, specifically covering the UAV's three-dimensional coordinates on the map, Euclidean distance to the target point, remaining fuel, and current time step. All data are normalized to a uniform dimensionless interval to ensure the consistency of network input data.
[0030] In step S1, the local terrain map is a 40×40 pixel local altitude matrix centered on the current position of the UAV, and three consecutive simulation steps are stacked to preserve the temporal terrain features; the flight state vector includes the three-dimensional coordinates of the UAV in the normalized map, the Euclidean distance to the target point, the remaining fuel amount, and the current time step.
[0031] S2. Based on a multimodal observation space, a highly normalized terrain height heatmap is used as the environmental representation method. First, three-dimensional terrain data of a fixed range around the current location of the UAV is acquired and mapped into a two-dimensional grid matrix with the same size as the local terrain map. The terrain height is mapped from the original physical space range to a dimensionless interval [0,1] using a linear normalization formula, where the maximum and minimum values of the original terrain height are determined according to the terrain parameters of the actual simulation environment. In the normalized terrain height heatmap, high-altitude areas correspond to bright values close to 1, and low-altitude areas correspond to dark values close to 0, which facilitates the subsequent extraction of terrain features by the convolutional neural network.
[0032] S3. Using the current 2D coordinates of the UAV as the center, construct a local observation window with the same size as the local topographic map, and set a fixed window radius. Construct an index matrix for the local grid using a grid generation function. The dimensions of the index matrix match the size of the local observation window, and the elements are consecutive integer sequences. Utilizing matrix broadcast addition, and combining the actual spatial distance corresponding to each pixel, generate a global coordinate matrix, ensuring that the elements of the global coordinate matrix are actual physical coordinates.
[0033] By utilizing the matrix broadcasting property, the global coordinate matrix and the global two-dimensional coordinates of the target point are calculated to obtain the square matrix of the Euclidean distance. This matrix is then substituted into the gravitational potential energy formula to control the rate at which gravitational potential energy decays with distance, generating a gravitational potential energy matrix consistent with the size of the local observation window.
[0034] The algorithm iterates through all threat sources in the environment (static obstacles such as mountains, buildings, and high-voltage lines, and dynamic interference sources such as pedestrians, vehicles, and birds), calculating the distance moments from the local grid to the center coordinates of each threat source in parallel based on global coordinate moments. Substituting the distance matrix into the repulsive potential energy formula generates the repulsive potential energy matrix corresponding to each threat source, and then summing the elements to obtain the local repulsive potential energy matrix.
[0035] Invalid grid points exceeding the physical boundaries of the map are filtered out using a boundary mask. The boundary mask value is set to 0 when the global coordinates of grid point Pij exceed the physical boundary range set by the simulation environment, and 1 otherwise. The local total potential field matrix is calculated and added to the local artificial potential field map in the multimodal observation space.
[0036] The global coordinate matrix is calculated in step S3 as follows:
[0037] Let the two-dimensional coordinates of the UAV at the current moment be . The index matrix of the local mesh is and Window radius is The global coordinate matrix is and .
[0038] The calculation process for the gravitational potential energy matrix in step S3 is as follows: First, utilize the matrix broadcasting property to broadcast the global coordinate matrix. and The square matrix of the Euclidean distance is obtained by performing calculations with the global two-dimensional coordinates of the target point. :
[0039] Substitute the squared matrix of the Euclidean distance into the gravitational potential energy matrix:
[0040] in, The target gravitational decay coefficient, used to control the rate of decay of gravitational potential energy, is a fixed parameter, ultimately yielding the gravitational potential energy matrix. ; The calculation process for the repulsive potential energy matrix is as follows: First, iterate through all threat sources in the environment, based on the global coordinate matrix. and Parallel computation from local mesh to various threat sources Distance matrix : Substitute the distance matrix into the repulsive potential energy matrix: in, For the environment The attenuation coefficient of the repulsive field of a threat source is a fixed parameter, which can be set to a larger value for high-risk threat sources such as high-voltage power lines and fast-moving vehicles. By triggering the evasion action in advance, the repulsive potential energy matrix is ultimately obtained. .
[0041] Local total potential matrix in step S3 The calculation formula is:
[0042] in, For the boundary mask, when grid points When outside the physical boundaries of the map Set the value to 0, and set it to 1 if it does not exceed the boundary. The values at the grid points corresponding to the repulsive potential energy matrix are... The values at the grid points corresponding to the gravitational potential energy matrix are denoted as .
[0043] S4. Construct a dual-flow collaborative sensing network containing terrain sensing flow and potential field sensing flow. Input the local terrain map into the terrain sensing flow to extract terrain geometric texture and physical boundary feature vectors. Input the local total potential field matrix into the potential field sensing flow to extract environmental hazard gradient and virtual safety passage feature vectors.
[0044] In step S4, the terrain perception stream and the potential field perception stream are computed in parallel. At the same time, features are extracted from the input local terrain map and the local total potential field matrix to ensure that the timeliness of the two types of features is consistent, providing synchronous environmental information support for subsequent feature fusion and decision output.
[0045] The terrain-aware flow network structure consists of two convolutional layers, inter-layer ReLU activation functions, and max-pooling layers. The first convolutional layer uses 32 3×3 kernels with a stride of 1 to extract primary texture features of the terrain in the local terrain map (such as subtle changes in terrain undulations, small bumps or depressions, etc.). The second convolutional layer uses 64 3×3 kernels with a stride of 1 to further extract the geometric contour features of complex terrains such as mountains, ravines, and steep slopes. Each convolutional layer is followed by a ReLU activation function to introduce nonlinear transformation to enhance the network's feature representation ability and avoid gradient vanishing. The output of the second convolutional layer is downsampled by a 2×2 max-pooling layer with a stride of 2, which reduces the data dimensionality and computational cost while preserving key obstacle features. Finally, the terrain-aware flow outputs a 16-dimensional high-level abstract feature vector through a fully connected layer. This vector effectively represents the physical boundaries and traversability information of the terrain, clearly identifying physical collision risk areas during UAV flight.
[0046] The network structure for potential field sensing flow consists of two convolutional layers, inter-layer ReLU activation functions, and max-pooling layers. The first convolutional layer uses 32 3×3 kernels with a stride of 1, focusing on extracting the basic gradient features of potential energy distribution in the local total potential field matrix. This includes the strength variation of potential energy between adjacent grid points, the preliminary outline of low-potential regions, and the dispersed distribution features of high-potential threat points, thus achieving an initial capture of the overall potential field distribution trend. The second convolutional layer uses 64 3×3 kernels with a stride of 1, used for in-depth mining of complex correlation features of the potential field, including the outline of virtual safety channels formed by continuous low-potential regions and the radiation range boundary of high-potential threat sources. The system collects key information such as the potential energy equilibrium point formed by the superposition of gravitational and repulsive fields. Each convolutional layer is followed by a ReLU activation function, which enhances the network's sensitivity to potential field gradient differences through nonlinear transformation, ensuring accurate differentiation of potential energy regions of different intensities. After the output of the second convolutional layer, it is downsampled through a 2×2 max pooling layer with a stride of 2, compressing the data dimension and improving computational efficiency while preserving the core potential energy distribution characteristics. Finally, the potential field perception stream outputs a 16-dimensional high-level abstract feature vector through a fully connected layer. This vector can effectively characterize the dangerous gradient distribution of the environment and the optimal navigation direction, giving the UAV an intuitive ability to predict potential flight paths.
[0047] S5. The terrain feature vector output by the terrain perception stream, the potential field feature vector output by the potential field perception stream, and the flight state vector in step S4 are concatenated into a joint feature vector. The dimension of the joint feature vector is the sum of the dimensions of the three types of vectors. This joint feature vector is then input into the PPO algorithm backend network, which includes a policy network (Actor) and a value network (Critic).
[0048] Policy Network: It contains two fully connected layers with a fixed hidden layer dimension. It outputs a fixed-dimensional action probability distribution vector, which corresponds to various discrete flight actions of the UAV, including hovering in place, moving along the positive and negative directions of the X, Y, and Z axes, and compound maneuvers. Each action corresponds to a fixed actual spatial movement distance or maneuver mode.
[0049] Value Network: It consists of two fully connected layers with a fixed hidden layer dimension. It outputs a state value scalar to evaluate the expected return of the current state and guide policy updates.
[0050] The UAV randomly samples the action probability distribution vector output by the policy network to generate and execute the specific flight action for the current step size.
[0051] S6. Based on the UAV's flight status, action execution results, and local total potential field matrix, a potential energy-driven composite reward function is used to calculate the reward value. By setting a low entropy coefficient for the PPO algorithm, the randomness of UAV action selection is suppressed. Combined with batch experience data and single-round sampling steps, the parameters of the PPO algorithm backend network and the dual-stream collaborative perception network are updated to iteratively optimize the path planning strategy.
[0052] The composite reward function in step S6 The specific expression is: Among them, the core reward items This reward is given positively when the drone enters a low-potential-energy region and negatively when it enters a high-potential-energy region. This represents the local total potential field value. Distance Guide Rewards This reward is used to provide continuous positive feedback as the drone moves towards the target point, even before reaching the destination. The distance to the target point at the current moment. The distance from the previous moment. This is the scaling factor, with a value range of 0.1-1.0; The mission completion reward is a large one-time reward given when the drone reaches the target point's defined range. The collision penalty is used to inflict a severe negative penalty and terminate the round when the drone touches a terrain boundary or enters the area of an obstacle entity; The time step penalty is used to incentivize the drone to find the shortest path to reduce flight time. High energy consumption penalty This penalty is used to suppress ineffective, sharp climbs or dives, ensuring a smooth and energy-efficient flight path. The current altitude. The altitude at the previous moment, This is the scaling factor, with a value range of 0.1-1.0.
[0053] The specific implementation of parameter update and iterative optimization in step S6 is as follows: The parameters of the PPO backend network and the dual-stream collaborative sensing network are bound into a unified parameter set, with a fixed total parameter size. From the path experience data accumulated during UAV environmental interaction, observation data acquisition, feature extraction, and action execution in steps S1 to S5, a batch of samples that can reduce noise interference from single samples and reflect the interaction patterns of multiple scenarios are selected. Combined with the reward value calculated by the potential energy-driven composite reward mechanism in step S6, the unified parameter set is synchronously gradient calculated and parameters are updated using an adaptive moment estimation algorithm (emphasizing a linear decaying learning rate strategy).
[0054] Each training round sets a fixed number of sampling steps, iteratively executing the process of "environmental interaction sampling - batch data extraction - parameter gradient update". During training, model parameters are saved periodically at a preset frequency, and the performance indicators of the path planning strategy (including task completion rate, collision rate, average flight time, and average energy consumption) are monitored and recorded in real time until the preset total number of training steps is completed, outputting the optimal path planning strategy with efficient obstacle avoidance, accurate arrival, and low energy consumption characteristics.
[0055] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A UAV 3D path planning method based on artificial potential field enhancement and reinforcement learning, characterized in that, Includes the following steps: S1. Construct a three-dimensional flight simulation environment, and define a UAV multimodal observation space based on the three-dimensional flight simulation environment, which includes a local topographic map, a local artificial potential field map to be generated, and flight state vectors; S2. Based on the multimodal observation space, map the three-dimensional terrain data around the current location of the UAV into a two-dimensional grid matrix and complete the normalization process; S3. Construct a local observation window centered on the current two-dimensional coordinates of the UAV and matching the size of the local topographic map, map and generate a global coordinate matrix, calculate the gravitational potential energy matrix and repulsive potential energy matrix in parallel, combine the boundary mask to generate a local total potential field matrix and supplement it to the multimodal observation space. S4. Construct a dual-stream collaborative sensing network containing terrain sensing flow and potential field sensing flow, and input the local terrain map and local total potential field matrix into the dual-stream collaborative sensing network to extract terrain geometric texture and physical boundary feature vectors, environmental hazard gradient and virtual safety passage feature vectors respectively; S5. The two types of feature vectors extracted in step S4 are concatenated with the flight state vector to form a joint feature vector, which is then input into the PPO algorithm backend network to output the action probability distribution corresponding to the UAV's discrete flight actions. The UAV performs the corresponding flight actions according to the action probability distribution. S6. Calculate the reward value based on the UAV's flight status, action execution results, and local total potential field matrix, update the parameters of the PPO algorithm backend network and the dual-stream collaborative sensing network, and iteratively optimize the path planning strategy.
2. The UAV 3D path planning method based on artificial potential field enhancement and reinforcement learning as described in claim 1, characterized in that: The three-dimensional flight simulation environment described in step S1 includes a terrain model, fixed target points, and static obstacles.
3. The UAV 3D path planning method based on artificial potential field enhancement and reinforcement learning as described in claim 1, characterized in that: The local terrain map mentioned in step S1 is a local height matrix centered on the current position of the UAV, and multiple consecutive simulation steps are stacked to preserve temporal terrain features; The flight state vector includes the UAV's three-dimensional coordinates in the normalized map, its Euclidean distance from the target point, remaining fuel, and current time step.
4. The UAV 3D path planning method based on artificial potential field enhancement and reinforcement learning as described in claim 1, characterized in that: The normalization process described in step S2 involves mapping the terrain height from physical space to a dimensionless interval and using a terrain height heatmap as an environmental representation method.
5. The UAV 3D path planning method based on artificial potential field enhancement and reinforcement learning as described in claim 1, characterized in that: The calculation method for the global coordinate matrix in step S3 is as follows: Let the two-dimensional coordinates of the UAV at the current moment be . The index matrix of the local mesh is and Window radius is The global coordinate matrix is and .
6. The UAV 3D path planning method based on artificial potential field enhancement and reinforcement learning as described in claim 5, characterized in that: The calculation process for the gravitational potential energy matrix in step S3 is as follows: First, utilize the matrix broadcasting property to broadcast the global coordinate matrix. and The square matrix of the Euclidean distance is obtained by performing calculations with the global two-dimensional coordinates of the target point. : Substitute the square matrix of the Euclidean distance into the gravitational potential energy matrix: in, The target gravitational decay coefficient, used to control the rate of decay of gravitational potential energy, is a fixed parameter, ultimately yielding the gravitational potential energy matrix. ; The calculation process for the repulsive potential energy matrix is as follows: First, iterate through all threat sources in the environment, based on the global coordinate matrix. and Parallel computation from local mesh to various threat sources Distance matrix : Substitute the distance matrix into the repulsive potential energy matrix: in, For the environment The repulsive field attenuation coefficient of a threat source is a fixed parameter, which can be set to a larger value for high-risk threat sources. By triggering the evasion action in advance, the repulsive potential energy matrix is ultimately obtained. .
7. The UAV 3D path planning method based on artificial potential field enhancement and reinforcement learning as described in claim 6, characterized in that, The local total potential field matrix mentioned in step S3 The calculation formula is: in, For the boundary mask, when grid points When outside the physical boundaries of the map Set the value to 0, and set it to 1 if it does not exceed the boundary. The values at the grid points corresponding to the repulsive potential energy matrix are... The values at the grid points corresponding to the gravitational potential energy matrix are denoted as .
8. The UAV 3D path planning method based on artificial potential field enhancement and reinforcement learning as described in claim 1, characterized in that: In step S4, the terrain-aware stream and the potential field-aware stream extract features synchronously using parallel computing. Both extract basic features through convolutional layers. After downsampling and compressing the data dimension, a high-level abstract feature vector is output by a fully connected layer. The feature vector output by the terrain-aware stream represents the physical boundary and traversability information of the terrain, while the feature vector output by the potential field-aware stream represents the dangerous gradient distribution and optimal navigation direction of the environment.
9. The UAV 3D path planning method based on artificial potential field enhancement and reinforcement learning as described in claim 1, characterized in that: The composite reward function described in step S6 The specific expression is: Among them, the core reward items , This represents the local total potential field value. Distance Guide Rewards , The distance to the target point at the current moment. The distance from the previous moment. This is the scaling factor; Rewards for completing the task. As a penalty for collision, Penalty for time step; High energy consumption penalty , The current altitude. The altitude at the previous moment, This is the scaling factor.
10. The UAV 3D path planning method based on artificial potential field enhancement and reinforcement learning as described in claim 1, characterized in that, The specific implementation of parameter update in step S6 is as follows: the parameters of the PPO backend network and the dual-stream collaborative sensing network are bound into a unified parameter set. Based on the reward value calculated by batch empirical data and composite reward function, the unified parameter set is synchronously gradient calculated and parameters are updated through an adaptive optimization algorithm. Training ends according to the preset total number of training steps or the convergence of policy performance, and the optimal path planning policy is output.