An apf unmanned aerial vehicle path planning method based on a pomdp model

Abstract

The application discloses an APF unmanned plane path planning method based on a POMDP model, uses a sensor carried by the unmanned plane to acquire information of an obstacle, establishes a partially observable Markov prediction model of the obstacle, and predicts a possible position of the obstacle in the next step by using the model; in order to ensure safety of the unmanned plane in path planning, an influence model of the unmanned plane and the obstacle is established to avoid collision between the unmanned plane and the obstacle in the process of path planning; the traditional artificial potential field method is improved to ensure that the unmanned plane can safely arrive at a target point and avoid being trapped in a local minimum value problem. The method has good feasibility and effectiveness in real-time path planning of the UAV, and can make the UAV effectively avoid the obstacle and find an optimal path.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) path planning technology, and specifically to an APF UAV path planning method based on the POMDP model. Background Technology

[0002] Unmanned aerial vehicles (UAVs) are powered, unmanned, and reusable aircraft. They are crucial tools for expanding human understanding and exploring the skies, capable of operating autonomously in places inaccessible to humans and performing tasks impossible for humans, thus possessing profound research value. Compared to manned aircraft, UAVs offer advantages such as smaller size, lighter weight, lower manufacturing costs, and simpler operation. With the rapid advancement of science and technology, UAVs are rapidly expanding in many application areas, including real-time monitoring, remote sensing, search and rescue, and precision agriculture. Research on UAVs has received widespread attention in recent years, and UAV path planning technology has become one of the most challenging and valuable research technologies in the aviation field.

[0003] Unmanned aerial vehicle (UAV) path planning technology is a cutting-edge research achievement integrating multiple disciplines such as mathematics, statistics, kinematics, and artificial intelligence. Its development has a significant impact on industries such as military, transportation, agriculture, and film. UAV path planning refers to setting a start and end point in the UAV's flight environment map and using relevant path planning methods to plan a collision-free, optimal, and safe flight path. Path planning is a key technology for improving the autonomous flight capability of UAVs and ensuring flight safety. A suitable path planning method is a prerequisite and foundation for the successful completion of UAV flight missions. There are many commonly used UAV path planning methods, such as genetic algorithms, particle swarm optimization, A-star algorithms, fast random search tree algorithms, Dijkstra's algorithm, and the Artificial Potential Field (APF) method.

[0004] Artificial potential field (APF) is a relatively simple path planning method. Due to its advantages such as simple structure, low computational cost, and high real-time performance, it is widely used in local path planning. Furthermore, APF is highly portable; by changing the source of the artificial potential field, it can also solve obstacle avoidance problems in multi-agent systems and terrain obstacle avoidance problems. However, existing APF methods also have some problems. Koren and Borrenstein pointed out four important inherent problems of the APF method. Among them, local minima are a problem that urgently needs to be solved.

[0005] Due to the high speeds of drones and moving obstacles in the air, relying solely on sensor detection is insufficient for real-time drone path planning. Therefore, it's necessary to further estimate the next movement position of obstacles based on data from several detected moments to ensure both the safety and real-time performance of path planning. Since the MDP model requires complete knowledge of the state space to solve for the optimal strategy, while the POMDP model is well-suited for uncertainties in the environment, actions, and observations, most current papers using the POMDP model for drone path planning focus on the drone as the central element. This can easily lead to frequent path switching by the drone, reducing path planning efficiency. Furthermore, most algorithms treat obstacles as particles during drone path planning, ignoring the influence of their actual shapes. This can easily result in collisions between the drone and obstacles, causing path planning failure. Summary of the Invention

[0006] This invention, by fully utilizing the path planning characteristics of UAVs, combines the partially observable Markov decision-making POMDP model with the improved artificial potential field method (APF), which can ensure the safety of UAV path planning in unknown environments.

[0007] To achieve the above objectives, this application proposes an APF (Automatic Path Planning) UAV path planning method based on the POMDP (Programmatical Object Model), comprising:

[0008] Step 1: Set the starting point and target point coordinates of the drone. The initial state of the drone is to fly in a straight line towards the target point.

[0009] Step 2: During the flight of the drone, detect surrounding obstacles using onboard sensors, acquire obstacle information using a gimbal, and establish an obstacle impact model;

[0010] Step 3: When the drone is detected to be within the obstacle influence model, the current position of the obstacle is obtained. The dynamic obstacle trajectory is predicted by the POMDP model to obtain the position of the obstacle at the next moment.

[0011] Step 4: Based on the current and next positions of the obstacle, model the repulsive force of the obstacle and find the set of potential fields with the lowest repulsive force in the potential field;

[0012] Step 5: Select the two positions that minimize the objective function D from the sets of lowest repulsive potential fields at the current time and the next time, respectively;

[0013] Step 6: Determine an arc based on these two positions and the drone's current position. The drone flies along this arc until the onboard sensors detect the surrounding environment information in the next time period (which can be set to an interval of 2 seconds).

[0014] Step 7: If obstacles are still detected in the obstacle influence model, proceed to step 3; if no obstacles are detected in the surrounding environment or are not in the obstacle influence model, the drone flies toward the target point until it reaches the target point.

[0015] Furthermore, dynamic obstacle trajectories are predicted using POMDP, specifically as follows:

[0016] Obtain the state space S, which includes two subsystem states: the motion state of the UAV. t and the motion state of obstacles t Therefore, the state space at time t is defined as:

[0017] S t =(UAV) t Obstacle t (1)

[0018] Among them, the motion state of UAV t This represents the position and velocity of the UAV at time t; it is represented by a five-dimensional vector. in This indicates the position of the drone at time t. This represents the speed of the drone at time t. This represents the direction of motion of the drone at time t; similarly, it represents the motion state of the obstacle. in This indicates the position of the obstacle at time t. This represents the velocity of the obstacle at time t. Indicates the direction of motion of the obstacle at time t;

[0019] Action A taken by the obstacle at time t t As a space for action:

[0020]

[0021] Among them, A t This refers to the action taken by the obstacle at time t. It refers to the distance that the obstacle has shifted in the geodetic coordinate system at time t relative to time t-1. This refers to the tilt angle relative to time t-1. This refers to the acceleration of the obstacle at time t;

[0022] The observation space is defined as the motion state of the obstacle observed by the UAV through its onboard sensors at time t. t :

[0023] Ob t ={obt |ob t ∈S t} (3)

[0024] Among them, Ob t Let be the obstacle's motion state observed by the airborne sensors at time t, including the obstacle's position, velocity, direction of motion, and azimuth angle θ relative to the UAV; the azimuth angle θ is obtained by the following formula:

[0025]

[0026] Furthermore, in the POMDP model, the observation space of obstacles is defined as an observation probability function in the presence of noise:

[0027] O(a t ,s t+1 ,o t+1 ) = Pro(o t+1 |s t+1 ,a t )+Err (5)

[0028] This formula indicates that when noise is present, the obstacle takes action a at time t. t Then, at time t+1, state s is reached. t+1 At that time, o was observed t+1 The probability of ; where a t ∈A t s t+1 ∈S t o t+1 ∈Ob t Err is the sensor's observation noise sequence, as shown in the following equation:

[0029] Err=k·Dis(UAV t Obstacle t )+m (6)

[0030] The observation probability function depends on the UAV's position relative to the obstacle, where k and m are fixed coefficients greater than 0, and Dis(UAV) t Obstacle t ) represents the distance between the drone and the obstacle at time t.

[0031] Furthermore, the state transition function of the UAV is:

[0032]

[0033] function This describes the process of formulating the state control dynamics equations for the UAV, using the following mapping relationship:

[0034]

[0035] The position of the UAV at time t+1 is obtained by equation (8); where α is the angle between the velocity direction of the UAV and the y_z plane, β is the angle between the velocity direction of the UAV and the x_y plane, and T is the sampling period between the two times.

[0036] The state transition function of the obstacle is defined as the state transition probability:

[0037] T(s t ,a t ,s t+1 ) = Pro(s t+1 |s t ,a t ,s t-1 ,a t-1 ,…s0,a0)=Pro(s t+1 |s t ,a t (9)

[0038] The formula represents the action a taken by the obstacle at time t. t Then, at time t+1, it reaches s. t+1 The probability of a state has the Markov property.

[0039] Furthermore, the POMDP model introduces belief states, which are the posterior probability distributions of each state, representing the confidence level in predicting the state. These belief states are updated using Bayes' theorem based on historical observations and action values.

[0040] B t+1 =Pro(S) t+1 |B t ,o t ,a t (10)

[0041] This formula indicates that the obstacle's belief state at time t is B. t Given the premise, choose action a t Then transition to the next belief state B t+1 The probability process.

[0042] Furthermore, the reward function is represented by the safety of the drone after taking action based on the obstacle state obtained by the onboard sensors at time t, and the number of times the drone's direction of motion changes. Specifically:

[0043] R(s,a)=R count (s,a)+R safe (s,a) (11)

[0044] If the drone collides with an obstacle, then R safe If (s,a) the reward is 0, path planning fails; if the drone can successfully reach the target point, then R... safe The security bonus for (s,a) is 100; R count (s,a) represents the number of times the drone changes its direction of motion after detecting an obstacle, R count For every 1 increment of (s,a), it means that the drone needs to adjust its direction of movement at that moment, and the reward is -10.

[0045] Furthermore, the environment is represented by a Cartesian rectangular grid for obstacle position prediction. Obstacle information measured by airborne sensors is mapped into the environmental coordinate system. In the grid coordinate system, an obstacle may fly to one of its eight surrounding flight heading angles or remain at its current position. The eight flight heading angles represent east, northeast, north, northwest, west, southwest, south, and southeast, respectively.

[0046] Furthermore, the obstacle influence model specifically utilizes a minimum cube to completely enclose the obstacle, with the diagonal length of the cube as the diameter of the circumscribed sphere of the obstacle.

[0047] As a further step, a virtual artificial repulsive field is established for the drone during its movement. A single obstacle forms a repulsive potential field, and multiple obstacles are equivalent to a repulsive potential field in a specific area. It is assumed that the repulsive potential field formed by a single obstacle is a sphere centered on the obstacle, and its influence range is determined by the obstacle influence model.

[0048] The repulsive potential field of obstacle Obs1 at point B is: The repulsive potential field of obstacle Obs2 at point B is: The resultant potential field formed by the two obstacles Obs1 and Obs2 is: The combined potential field generated by multiple obstacles at a certain point can be represented as:

[0049]

[0050] Where η is the repulsion coefficient, R o Let r be the radius of the sphere in the obstacle model. i Let r be the minimum safe flight distance for obstacle Obs1, and r be the distance between any point and the obstacle.

[0051] When a drone enters the sphere of influence, the repulsive force it experiences is represented as:

[0052]

[0053] Among them, R u Let be the radius of the sphere in the drone model.

[0054] As a further step, based on the obstacle information detected by the UAV in the unknown environment at the current moment, a repulsive equivalent potential field of the unknown environment at the current moment is established. Using the obstacle prediction information obtained from the POMDP model, a repulsive equivalent potential field of the unknown environment at the next moment is established. By combining the repulsive equivalent potential fields of the unknown environment at the current moment and the next moment, the lowest position of the potential field is obtained.

[0055]

[0056] The UAV's flight path is selected from the location of the lowest repulsive potential field, and its objective function is shown in equation (14); where N represents the set of coordinates of the lowest point of the repulsive potential field, and d 1i This refers to the distance d between the i-th point in set N and the reference route. 2i This refers to the distance d between the i-th point in set N and the location P of the drone. 3i It refers to the reciprocal of the distance between the i-th point in set N and the nearest obstacle.

[0057] Compared with existing technologies, the technical solutions adopted in this invention have the following advantages: This invention solves the problem that traditional artificial potential field methods can get stuck in local minima during UAV path planning, leading to target unreachability. Furthermore, it proposes for the first time an obstacle-centered POMDP model to predict the motion state of obstacles, improving the efficiency of UAV path planning. Based on a cubic circumscribed sphere model of obstacles, it can effectively avoid collisions between the UAV and obstacles, improving the safety of path planning. This method can ensure the safety of UAV path planning in unknown environments. Attached Figure Description

[0058] Figure 1 This is a two-dimensional obstacle model diagram proposed in this invention;

[0059] Figure 2 This is a diagram illustrating the obstacle influence model of the present invention;

[0060] Figure 3 This is a force analysis diagram of the present invention;

[0061] Figure 4 This is the equivalent potential field diagram of the present invention;

[0062] Figure 5 This is a schematic diagram of the analysis process of the present invention;

[0063] Figure 6 Comparison of drone trajectories across multiple static obstacles;

[0064] Figure 7 A magnified view of a portion of the drone trajectory comparison image with multiple static obstacles;

[0065] Figure 8 Comparison of drone trajectories for a single dynamic obstacle;

[0066] Figure 9 A magnified view of a portion of the drone trajectory comparison image for a single dynamic obstacle;

[0067] Figure 10 Comparison of drone trajectories with multiple dynamic obstacles;

[0068] Figure 11 This is a magnified view of a comparison of drone trajectories with multiple dynamic obstacles. Detailed Implementation

[0069] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit the application; that is, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0070] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0071] In the flight map of a drone, since the drone's environmental status is unknown and the surrounding environment is complex, the drone may encounter many typical obstacles during its flight from the starting point to the target, such as buildings, mountains, fire threats, and other flying objects. At this time, the drone needs to use its onboard sensors to obtain the distance and azimuth of static or dynamic obstacles in real time, and plan its flight path in real time based on these measurements to ensure it can successfully avoid obstacles and reach the target point. This invention assumes that the drone's flight speed is constant, and the drone initially flies towards the target point at a constant speed and direction. Therefore, it provides an APF drone path planning method based on the POMDP model, the key technical points of which are as follows:

[0072] Using the onboard sensors carried by the UAV to acquire obstacle information, a partially observable Markov decision process (POMDP) ​​model is established, and the model is used to predict the possible location of the obstacle in the next moment.

[0073] Specifically, this invention focuses on the path planning problem of unmanned aerial vehicles (UAVs) in unknown dynamic environments, where static and dynamic obstacles are randomly distributed in the flight environment. The purpose of this invention is to enable UAVs to successfully reach the target point while smoothly avoiding obstacles.

[0074] In the UAV path planning problem, the state space S contains two subsystem states: the motion state of the UAV and the state of the UAV. t and the motion state of obstacles t The state space at time t can be defined as:

[0075] S t =(UAV) t Obstacle t (1)

[0076] Among them, the status of UAVs t This represents the position and velocity of the UAV at time t; it is represented by a five-dimensional vector. in This indicates the position of the drone at time t. This represents the speed of the drone at time t. This represents the direction of motion of the drone at time t. Similarly, the motion state of the obstacle at time t can be represented. in, This represents the velocity of the obstacle at time t. Some data about the obstacle in path planning can be observed through airborne sensors, such as its position, velocity, and heading.

[0077] In path planning problems, a UAV can take corresponding actions based on the obstacle's motion state obtained from sensors, thereby controlling the UAV's state and changing its flight path. Therefore, this invention uses the action A taken by the obstacle at time t. t As a space for action:

[0078]

[0079] Among them, A t This refers to the action taken by the obstacle at time t. It refers to the distance that the obstacle has shifted in the geodetic coordinate system at time t relative to time t-1. This refers to the tilt angle relative to time t-1. It refers to the acceleration of the obstacle at time t.

[0080] The observation space is defined as the motion state of the obstacle observed by the UAV through its onboard sensors at time t. t ;

[0081] Ob t ={ob t |ob t ∈S t} (3)

[0082] Among them, Ob t Let be the obstacle's motion state observed by the sensor at time t, including the obstacle's position, velocity, direction of motion, and azimuth angle θ relative to the UAV; the azimuth angle θ can be obtained by the following formula:

[0083]

[0084] In the POMDP model, due to distance and surrounding environmental factors, the sensors carried by the UAV cannot accurately measure the state of the obstacle at time t. At any given moment, only noisy observations can be obtained. Therefore, the obstacle observation equation is defined as an observation probability function in the presence of noise:

[0085] O(a t ,s t+1 ,o t+1 ) = Pro(o t+1 |s t+1 ,a t )+Err (5)

[0086] This formula indicates that when noise is present, the obstacle takes action a at time t. t Then, at time t+1, state s is reached. t+1 At that time, o was observed t+1 The probability of . Where, a t ∈A t s t+1 ∈S t o t+1 ∈Ob t Err is the sensor's observation noise sequence.

[0087] During the process of acquiring the state of an obstacle, Err is related to the position between the UAV and the obstacle. When the UAV is close to the obstacle, the observation noise is low and the observation accuracy is high. When the distance is greater, the noise is higher, and the observation error also increases accordingly. The noise Err can be expressed as:

[0088] Err=k·Dis(UAV t Obstacle t )+m (6)

[0089] The observation function depends on the UAV's position relative to the obstacle, where k and m are constant coefficients greater than 0, and Dis(UAV) t Obstacle t The distance t represents the distance between the drone and the obstacle. The main reason for setting m is to avoid a collision between the drone and the obstacle when the distance is 0.

[0090] Due to the complex surrounding environment, the target information obtained by the UAV through the sensor also has measurement deviations. Therefore, this invention sets a large safety distance when the UAV first obtains obstacle-related information through the sensor to avoid collisions between the UAV and the obstacle due to sensor observation errors. The closer the distance, the lower the noise. The noise is dynamically adjusted according to formula (6), thereby automatically adjusting the flight distance between the UAV and the obstacle.

[0091] The state transition function represents the probability distribution of the state at the next moment after taking an action at the current moment. For the two subsystems, the UAV and the obstacle, in the state space defined in this invention, their corresponding state transitions are defined as follows:

[0092] The state transition function of the UAV is:

[0093]

[0094] function This involves designing the dynamic equations for the state control of the unmanned aerial vehicle (UAV); here, the mapping relationship used in this invention is as follows:

[0095]

[0096] The position of the UAV at time t+1 is obtained using equation (8). Here, α is the angle between the UAV's velocity direction and the y_z plane, β is the angle between the UAV's velocity direction and the x_y plane, and T is the sampling period between the two times.

[0097] The state transition function of an obstacle can be defined as the state transition probability, representing the action a that the obstacle takes at time t. t Then, at time t+1, it reaches s. t+1 The probability of a state exhibits the Markov property. According to the law of total probability, the state transition probability can be decomposed into:

[0098] T(s t ,a t ,s t+1 ) = Pro(s t+1 |s t ,a t ,s t-1 ,a t-1 ,…s0,a0)=Pro(s t+1 |s t ,a t (9)

[0099] Since the flight environment of a drone is not fully observable, belief states are introduced into the POMDP model. A belief state is the posterior probability distribution of each state, representing the confidence level in predicting the state, and exhibits Markov property. Its update is obtained through Bayes' theorem based on historical observations and action values:

[0100] B t+1 =Pro(S) t+1 |B t ,o t ,a t (10)

[0101] This formula indicates that the obstacle's belief state at time t is B. t Given the premise, choose action a t Then transition to the next belief state B t+1 The probability process.

[0102] The reward function represents the reward value obtained after taking an action in a certain state. It is an important criterion for predicting the next position of an obstacle and usually needs to be defined according to multiple criteria. Given that the goal of UAV navigation in path planning is to minimize the path length to the target point while avoiding obstacles, safety is the primary consideration, followed by the shortest path factor.

[0103] The reward function of this invention is represented by the safety of the UAV after taking action based on the obstacle state obtained by the sensor at time t, and the number of times the UAV changes its direction of motion. The reward function is:

[0104] R(s,a)=R count (s,a)+R safe (s,a) (11)

[0105] If the drone collides with an obstacle, then R safe If (s,a) the reward is 0, path planning fails; if the drone can successfully reach the target point, then R... safe The security bonus for (s,a) is 100. R count (s,a) represents the number of times the drone changes its direction of motion after detecting an obstacle, R count For every 1 increment of (s,a), it means that the drone needs to adjust its direction of movement at that moment, and the reward is -10.

[0106] The environment is represented using a Cartesian rectangular grid for obstacle position prediction. Obstacle information measured by sensors can be mapped onto the environmental coordinate system. In this grid coordinate system, an obstacle may fly to one of its eight surrounding heading angles or remain at its current position. The eight heading angles represent east, northeast, north, northwest, west, southwest, south, and southeast, respectively.

[0107] By configuring the above components and using the grid map method, the POMDP model proposed in this invention can accurately predict the next position Ret of the obstacle, complete the trajectory generation of the obstacle, and lay a foundation for UAV path planning.

[0108] To ensure the safety of drones in path planning, an impact model of drones and obstacles is established to avoid collisions between drones and obstacles during the path planning process.

[0109] During flight, drones must consider not only their own constraints but also the threats posed by various obstacles. Most path planning algorithms treat the drone and various obstacles as particles, but in reality, obstacles are often irregular three-dimensional objects, such as birds or hills. Simply treating them as particles increases the risk of path planning problems. Since most static or dynamic obstacles in the air approximate spheres, such as hot air balloons, bats, and birds, this invention primarily handles spherical or spherical obstacles. Any obstacle whose length-to-width ratio is between 5:2 can be approximated as a sphere.

[0110] To ensure the generation of the drone's flight path and its safety when encountering obstacles, this invention proposes a nested minimum cube circumscribed sphere obstacle model. Specifically, the improved method involves detecting the obstacle's image information using the drone's gimbal, completely enclosing the obstacle with a minimum cube, and using the diagonal length of the cube as the diameter of the obstacle's circumscribed sphere. The two-dimensional obstacle model is shown in the attached figure. Figure 1 As shown in the diagram. The innermost solid line represents a two-dimensional planar view of the obstacle, the middle dashed line represents a two-dimensional model of a cube, and the outermost solid line represents a two-dimensional model of the circumscribed sphere.

[0111] During path planning, both the drone and the obstacle are assumed to be spheres, with their respective centers. An obstacle influence model is used to design a safe flight path range for the drone, providing further assurance for obstacle avoidance. The obstacle influence model is attached. Figure 2 As shown. Where R u R represents the radius of the sphere containing the drone. o The radius of the sphere representing the obstacle is given. This is conditional on the condition that the distance between the center of the drone's sphere and the center of the obstacle's sphere is greater than the sum of their radii, i.e., dis > Dis(R). u ,R o If a path is considered safe and effective, the drone can safely reach its destination by following this path. Here, dis represents the distance between the center of the drone's sphere and the center of the obstacle's sphere.

[0112] An improvement to the traditional artificial potential field method is made to ensure that the UAV can safely reach the target point while avoiding it from getting trapped in local minima.

[0113] As a drone travels to its target point, there may be a point where the gravitational and repulsive forces are equal, causing the drone to become stuck and unable to move. This is called a local minimum. At this point, the net repulsive force on the drone is equal in magnitude and opposite in direction to the gravitational force.

[0114] A virtual artificial repulsive field is created for the drone during its movement. A single obstacle forms a repulsive potential field, and multiple obstacles are equivalent to a repulsive potential field in a specific region. The force analysis is shown in the attached figure. Figure 3 As shown. It is assumed that the repulsive potential field formed by a single obstacle is a sphere centered on the obstacle, and the range of its influence is determined by the obstacle influence model.

[0115] The repulsive potential field of obstacle Obs1 at point B is: The repulsive potential field of obstacle Obs2 at point B is: The resultant potential field formed by the two obstacles Obs1 and Obs2 is: Analysis shows that the combined potential field generated by multiple obstacles at a certain point can be expressed as:

[0116]

[0117] Where η is the repulsion coefficient, R o Let r be the radius of the sphere in the obstacle model. i denoted as , where is the minimum safe flight distance for obstacle Obs1, and r is the distance between any point and the obstacle.

[0118] When a drone enters the sphere of influence, the repulsive force it experiences is represented as:

[0119]

[0120] Among them, R u Let be the radius of the sphere in the drone model.

[0121] Based on the above analysis, it can be seen that multiple obstacles in an unknown environment form an equivalent potential field, as shown in the attached figure. Figure 4 As shown, the potential field is a virtual, special entity within the environment surrounding a moving obstacle. Its main function is to exert a repulsive force on drones entering it. The magnitude of the potential field is determined by the radius of the obstacle model and the distance between the drone and the obstacle. The direction of the potential field is always from the center of the obstacle towards the drone. Introducing this new repulsive force prevents the drone from flying to a local minimum point, fundamentally avoiding the occurrence of local minima.

[0122] Using the drone's starting position to target position as the reference flight path l, an artificial potential field is constructed in the unknown flight environment. The position coordinates of obstacles can be obtained from sensor measurements and the trajectory model of obstacles using the POMDP model.

[0123] Based on obstacle information detected by the UAV in the unknown environment at the current moment, a repulsive equivalent potential field of the unknown environment at the current moment is established. Using obstacle prediction information obtained from the POMDP model, a repulsive equivalent potential field of the unknown environment at the next moment is established. By combining the repulsive equivalent potential fields of the unknown environment at the current moment and the next moment, the lowest position of the potential field is obtained.

[0124]

[0125] To reduce energy consumption during path planning, this invention selects locations within the set of lowest repulsive potential fields as the planned paths. The UAV's flight path is selected from these lowest repulsive potential field locations. The selection of these locations is related to their distances to the reference flight path, obstacles, and the UAV itself. Here, N represents the set of coordinates of the lowest points in the repulsive potential field, and d... 1i This refers to the distance d between the i-th point in set N and the reference route. 2i This refers to the distance d between the i-th point in set N and the location P of the drone. 3i It refers to the reciprocal of the distance between the i-th point in set N and the nearest obstacle, and its objective function is shown in formula (14).

[0126] Figure 6 and Figure 7 This is a flight trajectory diagram of a drone in a multi-static obstacle environment. Figure 6 It can be seen that when the UAV starts moving, the paths planned by the four algorithms are all normal flight paths. When the onboard sensor detects obstacle obs2, the TAPF algorithm plans a path that goes around obs2 from above, while the other three algorithms choose to fly around it from below. From the UAV's starting point (0,0,0) to point (31,28,30), the Improve APF-fuzzy algorithm, the CPFIBA algorithm, and the POMDP-APF strategy of this invention are almost identical, and all outperform the TAPF algorithm. However, when the sensor detects the presence of obs3 and obs4, both the TAPF algorithm and the Improve APF-fuzzy algorithm exhibit large turning angles, and the TAPF algorithm gets stuck in a local minimum when encountering obs5, causing the path planning to fail. Although the Improve APF-fuzzy algorithm can complete the path planning, the simulation results show that it is far less efficient than the POMDP-APF strategy proposed in this invention. Figure 7A close-up view clearly shows that, compared to the Improve APF-fuzzy algorithm, the CPFIBA algorithm and the POMDP-APF strategy produce more complete obstacle avoidance paths without oscillations or significant turns. Furthermore, the POMDP-APF strategy generates shorter paths compared to the other three algorithms, providing safer obstacle avoidance.

[0127] Figure 8 and Figure 9 This is a flight trajectory diagram of a drone in a single dynamic obstacle environment. From... Figure 8 As can be seen, at the obs1 location, all three algorithms can effectively bypass the obs1 obstacle. Compared to the TAPF algorithm and the Improve APF-fuzzy algorithm, the POMDP-APF strategy has a smaller turning angle and path length when bypassing obs1. After the drone bypasses obs1, the obstacle starts moving from a stationary state. Figure 8 The dashed line in the diagram represents the obstacle trajectory predicted using the POMDP model. After a period of time, the obstacle moves from position obs1 to position obs1'. At this point, neither the TAPF algorithm nor the Improve APF-fuzzy algorithm can change the UAV's direction of movement in time, causing the TAPF algorithm to collide with the obstacle at position (75,77,90) and the Improve APF-fuzzy algorithm to collide at position (83,78,90), resulting in path planning failure. In contrast, the POMDP-APF strategy proposed in this invention continuously predicts the obstacle's next movement position in real time upon initial obstacle detection. If it determines that the UAV is outside the obstacle's influence range, it chooses to proceed directly to the target point without obstacle avoidance.

[0128] Figure 10 and Figure 11 The flight trajectories of the UAV under multiple dynamic obstacle environments are shown in Table 1. Simulations comparing the POMDP-APF strategy, TAPF algorithm, and Improve APF-fuzzy algorithm under multiple dynamic obstacle conditions are performed, with parameter settings as shown in Table 1. Here, obs2 and obs3 are static obstacles, while obs1 and obs4 are initially stationary. Once the UAV detects their presence through sensors and bypasses them, obs1 and obs4 are set to a moving state. Figure 10Simulation results of UAV obstacle avoidance under multiple dynamic obstacle conditions are shown. Compared with the TAPF method and the Improve APF-fuzzy algorithm, the POMDP-APF strategy proposed in this invention can effectively avoid all obstacles. The TAPF algorithm can complete the obstacle avoidance task when obs1 is stationary, but when obs1 is moving, it cannot perform dynamic obstacle avoidance and collides with obs1 at (18,54,40), causing path planning failure. The simulation results of the Improve APF-fuzzy algorithm are better than those of the TAPF algorithm. It can select a correct path to avoid the stationary obs1, but a collision occurs at (72,86,62) when obs4 starts moving. Figure 11 This is a magnified view of the obstacle obs4 section. Throughout the obstacle avoidance process, the POMDP-APF strategy ensures the drone obtains a relatively smooth path, regardless of whether static or dynamic obstacles are detected. Furthermore, during real-time obstacle avoidance, the drone's planned path is complete and free of oscillations.

[0129] Table 1 Example Parameter Settings

[0130]

[0131] The foregoing description of specific exemplary embodiments of the invention is for illustrative and explanatory purposes. These descriptions are not intended to limit the invention to the precise forms disclosed, and it will be apparent that many changes and variations can be made in accordance with the foregoing teachings. The exemplary embodiments were chosen and described in order to explain the specific principles of the invention and its practical application, thereby enabling those skilled in the art to implement and utilize various different exemplary embodiments of the invention, as well as various different choices and variations. The scope of the invention is intended to be defined by the claims and their equivalents.

Claims

1. An APF (Automatic Path Planning) UAV path planning method based on the POMDP (Programmatical Object Model), characterized in that, include: Step 1: Set the starting point and target point coordinates of the drone. The initial state of the drone is to fly in a straight line towards the target point. Step 2: During the flight of the drone, detect surrounding obstacles using onboard sensors, acquire obstacle information using a gimbal, and establish an obstacle impact model; Step 3: When the drone is detected to be within the obstacle influence model, the current position of the obstacle is obtained. The dynamic obstacle trajectory is predicted by the POMDP model to obtain the position of the obstacle at the next moment. Step 4: Based on the current and next positions of the obstacle, model the repulsive force of the obstacle and find the set of potential fields with the lowest repulsive force in the potential field; Step 5: Select from the sets of lowest repulsive potential fields at the current time and the next time respectively, the ones that make the objective function... The two smallest positions; Step 6: Determine an arc based on these two positions and the drone's current position. The drone flies along this arc until the onboard sensors detect information about the surrounding environment in the next time period. Step 7: If obstacles are still detected in the obstacle influence model, proceed to step 3; if no obstacles are detected in the surrounding environment or are not in the obstacle influence model, the drone flies toward the target point until it reaches the target point; Predicting dynamic obstacle trajectories using POMDP, specifically: Obtain the state space S, which includes two subsystem states: the motion state of the UAV and the state space of the UAV. and the motion state of obstacles t Therefore, the state space at time t is defined as: S t = (UAV t , Obstacle t ) (1) Among them, the motion status of the drone This represents the position and velocity of the UAV at time t; it is represented by a five-dimensional vector. ,in This indicates the position of the drone at time t. This represents the speed of the drone at time t. This represents the direction of motion of the drone at time t; similarly, it represents the motion state of the obstacle. ,in This indicates the position of the obstacle at time t. This represents the velocity of the obstacle at time t. Indicates the direction of motion of the obstacle at time t; Actions taken by the obstacle at time t As a space for action: （2） in, This refers to the action taken by the obstacle at time t. It refers to the distance that the obstacle has shifted in the geodetic coordinate system at time t relative to time t-1. This refers to the tilt angle relative to time t-1. This refers to the acceleration of the obstacle at time t; The observation space is defined as the motion state of the obstacle observed by the UAV through its onboard sensors at time t. : Ob t = {ob t | ob t ∈ S t} (3) in, Let t be the obstacle's motion state as observed by the airborne sensors, including the obstacle's position, velocity, direction of motion, and azimuth relative to the UAV. azimuth Obtained from the following formula: （4）。 2. The APF UAV path planning method based on the POMDP model according to claim 1, characterized in that, In the POMDP model, the observation space of obstacles is defined as an observation probability function in the presence of noise: The t ,s t+1 ,the t+1 )=Pro(o t+1 |s t+1 ,the t )+Err(5) This formula indicates that when noise is present, the obstacle takes action a at time t. t Then, at time t+1, state s is reached. t+1 At that time, it was observed The probability of ; where a t ∈A t s t+1 ∈S t o t+1 ∈Ob t Err is the sensor's observation noise sequence, as shown in the following equation: Err=k·Dis(UAV t ,Obstacle t )+m(6) The observation probability function depends on the position of the UAV relative to the obstacle, where , It is a fixed coefficient greater than 0, Dis(UAV) t Obstacle t ) represents the distance between the drone and the obstacle at time t.

3. The APF UAV path planning method based on the POMDP model according to claim 2, characterized in that, The state transition function of the UAV is: （7） function This describes the process of formulating the state control dynamics equations for the UAV, using the following mapping relationship: （8） The position of the UAV at time t+1 is obtained by equation (8); where, It is the angle between the drone's velocity direction and the y_z plane. It is the angle between the velocity direction of the UAV and the x_y plane, and T is the sampling period between the two moments; The state transition function of the obstacle is defined as the state transition probability: （9） The formula represents the action taken by the obstacle at time t. Then, it arrives at time t+1. The probability of a state has the Markov property.

4. The APF UAV path planning method based on the POMDP model according to claim 1, characterized in that, The POMDP model introduces belief states, which are the posterior probability distributions of each state, representing the confidence level in predicting the state. These belief states are updated using Bayes' theorem based on historical observations and action values. B t+1 =Pro(S t+1 |B t ,o t ,a t ) (10) This formula represents the belief state of the obstacle at time t. Given the premise, choose action a t Then transition to the next belief state The probability process.

5. The APF UAV path planning method based on the POMDP model according to claim 1, characterized in that, The reward function is represented by the safety of the drone after taking action based on the obstacle state obtained by the onboard sensors at time t, and the number of times the drone's direction of motion changes. Specifically: R(s,a)=R count (s,a)+R safe (s,a) (11) If the drone collides with an obstacle, then R safe If (s,a) the reward is 0, path planning fails; if the drone can successfully reach the target point, then R... safe The reward for (s,a) is 100; R count (s,a) represents the number of times the drone changes its direction of motion after detecting an obstacle, R count For every 1 increment of (s,a), it means that the drone needs to adjust its direction of movement at that moment, and the reward is -10.

6. The APF UAV path planning method based on the POMDP model according to claim 1, characterized in that, The environment is represented by a Cartesian rectangular grid for obstacle position prediction. Obstacle information measured by airborne sensors is mapped into the environmental coordinate system. In the grid coordinate system, an obstacle may fly to one of its eight surrounding flight heading angles or remain at its current position. The eight flight heading angles represent east, northeast, north, northwest, west, southwest, south, and southeast, respectively.

7. The APF UAV path planning method based on the POMDP model according to claim 1, characterized in that, The obstacle influence model specifically utilizes a minimum cube to completely enclose the obstacle, with the diagonal length of the cube as the diameter of the circumscribed sphere of the obstacle.

8. The APF UAV path planning method based on the POMDP model according to claim 1, characterized in that, A virtual artificial repulsive field is established for the drone during its movement. A single obstacle forms a repulsive potential field, and multiple obstacles are equivalent to a repulsive potential field in a specific area. It is assumed that the repulsive potential field formed by a single obstacle is a sphere centered on the obstacle, and its influence range is determined by the obstacle influence model. obstacle At point The repulsive potential field is ,obstacle At point The repulsive potential field is ,but and The combined potential field formed by the two obstacles is The combined potential field generated by multiple obstacles at a certain point can be represented as: （12） in, The repulsion coefficient is... Let be the radius of the sphere in the obstacle model. Obstacles The minimum safe flight distance, Let be the distance between any point and the obstacle; When a drone enters the sphere of influence, the repulsive force it experiences is represented as: （13） in, Let be the radius of the sphere in the drone model.

9. The APF UAV path planning method based on the POMDP model according to claim 1, characterized in that, Based on obstacle information detected by the UAV in the unknown environment at the current moment, a repulsive equivalent potential field of the unknown environment at the current moment is established. Using obstacle prediction information obtained from the POMDP model, a repulsive equivalent potential field of the unknown environment at the next moment is established. By combining the repulsive equivalent potential fields of the unknown environment at the current moment and the next moment, the lowest position of the potential field is obtained. （14） The UAV's flight path is selected from the position of lowest repulsive potential field, and its objective function is shown in equation (14); where, This represents the set of coordinates of the lowest point of the repulsive potential field. Refers to a set The first in The distance between each point and the reference route Refers to a set The first in The distance between each point and the location P of the drone Refers to a set The first in The reciprocal of the distance between each point and the nearest obstacle.

Images